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Chapter  1 
INTRODUCT IOU 

Problem  Statement 

ptf\fA  ^he  Department  of  Defense  (Dob)7  Cost/Schodule  Con- 
'  /  v 

trol  System  Criteria  (C/SCSC)^  was,  at  this  v/riting,*  still  a 

relatively  new  concept  for  control  of  management  information 

flow  between  DoD  components  and  major  defense  contractors. 

Since  it  "was, new  and  represented  a  substantial  departure  in 

both  philosophy  and  form  from  previously  used  systems,  a 

serious  lack  of  educational  material  existed.  Speeirraccily# 

there  was  no  material  available  of  the  self-contained  aose 

study  type  which  was>of  a  length  reasonable  :.or  use  \n  a 

graduate  level  course^in  which  the  intention  was  for  the 

case  to  serve ^as  an  overview,  rather  t:ian  as  a  detailed 
ry;  «  i  *  *.  X  K<CepfJ  .  ^  ' 

study  of  C/SCSC.  •  1  *  ,  ,  1  -sc  -.0  <-'}  '/  f' 

, ,  ^  >  >y  »•  c  do  C?1  c  ^  (  .  .  .  c  ,  X . ,  • ,  S  /,  >  ^ 

cl  fi  t  rc'r>fh£fc£he' studies  available  wore  without  exception 
fictitious,  rather  than  having  any  basis  in  an  actual 
application  of  C/S0.1C,  and  were  exceptionally  lore  thy,  con¬ 
taining  throe-hundred  to  six-hundred  written  pager.  This 
length  v;as  considered  excessive  for  use  as  a  ore  oay  over¬ 
view  of  C/f>CDC.  Puri  her,  the  available  carer.  v:<  rc  rot 
sedf --contained,  but  required  subst  antial  si.  0.< 
st£ ucl" ions  and  guidance  from  an  instructor. 


.  v.<u 


1 


Background 
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PuFinj  the  two  docodou  ij ubs cguont  to  nOiricl  t3x*^  XX 
f'pid  technological  chongc  coiubincci  v/ifh  bhc  roguizrorncnt 
brought  on  by  the  Cold  V.’ar  to  maintain  a  standing  military 
force  to  produce  a  rapidly  expanding  defense  industry. 
Contracts  between  DoD  corriponents  and  the  dcronse  industry 
soon  began  to  run  into  the  millions  and  hundreds  of  millions 
of  dollars.  This  growth  was  accompanied  by  larger  and  larger 
defense  budgets  and  by  increasing  demands  for  bettor  manage¬ 
ment  of  tne  defense  dollar,  especially  in  these  arcus  where  . 
a  relatively  small  management  error  could,  cost  the  taxpayers 
millions  of  dollars. 

In  response  to  increased  demands  for  better  manage¬ 
ment  and  increased  visibility  in^o  the  uses  of  ta;.  dollars 
on  large  contracts  a  number  of  management  information  sys¬ 
tems  v/ere  developed*  Disastrous  cost  over-runs  and  schedule 
slippages  v/ere  occurring  with  increasing  regularity,  host 
embarrassing  to  DoD  officials  v/as  the  fact  that  many  of 
these  over— runs  did  not  even  become  known  until  af  cor  the 
fact.  By  this  time  program  development  was  often  so  far 
along  that  little  choice  remained  but  to  obtain  missive 
additional  financing  in  order  to  receive  some  concrete 
return  on  dollars  already  invested. 

Public  criticism  of  Dol)  financial  management  placed 
increased  emphasis  on  developing  a  management  information 
system  which  v/ouid  beep  1X>D  .informed  of  the  status  of  major 
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programs  but,  at  the  same  time,  would  not  preempt  the  man¬ 
agement  prerogatives  of  contractors.  In  trying  various 
systems.  Dob  had  often  imposed  a  system  on  contractors. 
Further,  for  lack  of  central  control,  contractors  often 
ended  up  maintaining  several  costly  and  often  overlapping 
systems  on  various  DoD  contracts.^ 

One  of  the  many  systems  developed  during  this  period 
was  the  Program  Evaluation  and  Review7  Technique  (PERT)  which 
was  developed  by  the  Navy  in  1958  for  use  on  the  Polaris 
Missile  project.  Associated  v/ith  this  system  v*c  a  TFork 
Breakdown  Structure  (WBS)  that  subdivided  the  contract  work 
program  through  successive  levels  of  detail.  This  sub¬ 
division  was  carried  down  to  discrete  work  packages  which 
could  then  be  used  as  a  basis  for  planning  and  controlling 
work  effort  (schedule)  and  the  amount  expended  on  each 
package  (cost)  .  This  basic  system  was  adopted  by  Air  Force 
Systems  Command  (Al'SC)  in  1962  for  use  on  the  TFX  program  and 
was  expanded  to  include  an  earned  value  concept.  The  major 
departure  of  the  new  Air  Force  system  from  previous  ones 
was  that  it  did  not  attempt  to  spell  out  in  detail  the 
management  information  system  to  be  used  by  the  contractor. 
Instead,  it  specified  categories  of  information  that  vrere  to 
be  avail  able  from  the  system  and  left  jt  to  the  contractor 
to  design  and  implement  an  individual  system  that  would 


IT.  Ram  ford,  "The  Mont  ci  oy  Critique,**  Armed  Forces 
Management  ,  Vo l  O,  to  1  (Oclo':>  r,  3962),  pp.  *'2-60. 
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satisfy  both  the  contractor 1  s  needs  and  those  of  the  Go /em¬ 
inent. 

Based  on  the  demons t rated  success  of  these  efforts 
the  A FSC  published  its  Cost/Schedule  Planning  and  Control 
Specification  in  June  19G6.  This  system  incorporated  the 
better  techniques  from  both  PERT  and  earned  value  end  also 
included  implementation  of  recommendations  made  after 
further  studies  by  a  management  consultant  firm. ^ 

In  December  1967,  DoD  Instruction  7000.2  was  issued 
under  the  title  "Cost/Schedule  Conti ol  oys^-em  Criteria. 

This  instruction  adopted  7iFSC's  system  for  DcD-wice  use  to 
partially  fulfill  the  Resource  Management  System  requirement 
with  respect  to  management  of  major  capital  acquisitions. 

Resource  Management  Systems  (RMS)  is  the  name  given 
to  the  program  begun  by  Secretary  of  Defense  Robert  S. 
McNamara  in  August  1966  with  the  issuance  of  BoD  Directive 
7009.1.  The  basic  objective  of  RMS  was  improved  financial 

management  throughout  all  phases  of  resource  acquisi cions, 

2 

utilization,  and  retirement.  To  accomplish  this  objec¬ 
tive  four  major  interlock .i  ng  systems  were  created:  (1)  the 

Programming  and  Budgeting  System,  concerned  with  planning 
the  resources  needed  to  meet  DoD  objectives  nnd  with 
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lJ.  Jlonold  Fox  nnd  W.  K.  Carin',  "Cord:  Pennine- 
and  Control  in  V.’onponr.  7vnqnir.it  ioi  ,  "  A*.  d  r»  rccr  Cor-td  rol- 
ler,  (June,  1969),  pp.  9-11. 

'RJ.G.  Department  of  Defense,  i)i  nvi  ivc  Number  7090.1, 

Ro  sourer*  f  t  «  n;  uy  v.  ■  ten  I.  Sy : » t_c  •»  i»_‘ }_  of  fho  f  1  **  'f  ’  li °f  P:  on  sc, 

August  22,  196i>,  p.  3. 
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justifying  those  needs  to  Congress,  (2)  the  Operations 
Management  System,  concerned  with  management  of  resources 
directly  applied  to,  or  in  support  of,  the  execrating  com¬ 
mands  in  boD,  (3)  the  Inventory  Management  System  (IMS), 
concerned  with  management  of  items  in  the  DoD  supply  system, 
and  (4)  the  Acquisition  Information  and  Management  System* 
concerned  with  management  of  items/services  from  outside 
the  DoD.  Of  the  four,  the  latter  tv:o  dealt  specifically 
with  management  of  assets.  IMS  represented  the  internal 
portion  of  the  asset  management  structure  and  AIMS,  the 
external. 

AIMS  is  further  divided  into  two  major  subsystems: 
Selected  /acquisition  Information  Management  System  (SAIMS) 
and  Other  Capital  Acquisitions.  SAIMS,  in  turn,  consists  of 
the  Economic  Information  System,  Cost  Information  Reports, 
and  Contract  Fund  Status  Heports,  each  of  which  provide 
financial  information  by  levying  rejporting  requirements  on 
the  contractor,  and,  finally,  the  Performance  Measurement 
System  v/hich  integrates  the  cost,  schedule,  and  technical 
progress  under  the  contract.  Of  the  three  cnly  cost  and 
schedule  are  controlled  by  the  C/SCSC  Criteria. 

Scope 

'ihe  case  study  developed  from  this  effort  was 
specifically  intended  to  be  of  use  a::  an  educational  aid 
in  presenting  C/CCSC  to  a  group  composed  pri  warily  of 
Air  Force  officers.  Further,  the  Financial  Management 
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course  in  which  the  case  was  primarily  intended  to  be  used 
concentrated  on  Air  Force  applications  throughout  the  RMS 
presentation.  It  was  therefore  determined  that  the  case 
should  be  based  on  an  Air  Force  application  of  C/SCSC. - 

In  view  of  severe  time  and  travel  restrictions,  it 
v/as  further  decided  to  limit  the  choice  of  possible  programs 
to  those  in  which  the  Systems  Program  Office  (SPO)  was 
phvsirally  located  at  V»7right-Patterson  AFB,  Ohio.  It  was 
also  considered  desireable  to  chcose  a  program  in  which 
the  C/SCSC  application  had  progressed  beyond  the  validation 
stage  to  insure  availability  of  as  much  factual  background 
as  possible. 

It  was  also  determined  that  the  program  chosen  as 
the  basis  for  the  case  should  not  nave  experienced  unusu¬ 
ally  severe  problems  or  delays  due  to  the  O/SCSC,  since 
the  case  was  intended  to  demonstrate  how  the  system  should 
worlc,  based  on  a  successf  ul  a ppl i c a t ion • 

The  SPO  which  best  met  these  qualifications  was  the 
B-l  Manned  Bomber  SIX).  The  B-l  Program  was  therefore  chosen. 
To  further  limit  the  size  and  complexity  of  the  resultant 
case  it  was  determined  it  should  be  based  solcy  cn  the  major 
efforts  and  events  involved  in  validation  and  surveillance 
of  the  c/SCSC  effort  of  1  ho  prime  contractor,  Korth  /.merican 
Aviation  Division  of  north  American  Koohwoll. 
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Objective 

The  primary  objective  of  this  paper  was  to  develop 
a  ca.se  study  based  on  the  C/d  CSC  experience  of  the  B-l  SPO. 
The  resultant  case  was  designed  to  be  a  self-contained 
teaching  aid  appropriate  for  use  as  a  one  day  overview  of 
C/SCSC  for  Air  Force  officers  with  limited  prior  knowledge 
of  the  criteria. 

To  meet  the  overall  objective,  the  following  sub- 

objectives  were  identified: 

1.  To  present,  within  the  case  context,  sufficient 
discussion  of  C/GCSC  and  SAIKS  to  impart  a  basic  understand¬ 
ing  of  the  Criteria  and  of  the  environment  in  which  they  are 
used. 

2.  To  show  through  examples  drawn  from  the  experi- 
ence  of  the  D-3  SPO  how  one  application  of  the  Criteria  has 
v/orbed. 

3*  To  develop  o  series  of  questions  and  suggested 
answers  for  them,  each  based  on  experiences  related  in 
the  case  and  designed  to  generate  discussion  «-nd  thought 


on  some  specific  aspect  of  c:/SCSC. 

4.  To  develop  and  administer  a  test  applicol  ton  of 
the  case  to  a  selected  sample  ot  officers  in  oraci  to  verify 
its  usefulness  as  a  teaching  aid. 
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The  following  rr^cmrrli  questions  and  hypothesis 
wore  addressed  in  accompl  i shi ncj  the  stated  objectives  of 
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ResGd rch  q nor:  1: 1  on 3 .  Three  research  quest  ions  were 
developed : 

1.  Can  the  Cost/Scheciule  Control  System  Criteria 
be  presented  in  a  comprehensive,  self -cents j  nod  case  study? 

2.  Can  such  a  case  study  be  developed  based  on  an 
actual  C/SCSC  application? 

3.  Would  such  a  case  study  be  useful  in  imparting 
basic  understanding  of  the  C/SCSC  system,  methods,  and  pro¬ 
cedures? 

Hypothesis ,  In  connection  with  research  question 
number  three-  the  following  hypothesis  v;o:-  tested: 

A  sample  group  t.:  officers  v:ho  had  used  the  case 
study  would  score  significantly  higher  on  an  examination 
consisting  of  questions  pertaining  to  C/3CSC  and  SAKS  than 
v/ould  a  similar  group  who  had  the  subject  presented  to  them 
through  a  classroom  lecture. 
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Chapter  2 


PROCEDURE 

Pat, vi re  and  So'j rces  of  Data 

The  data  required  to  meet  the  objectives  of  this 
paper  fell  naturally  into  tv;o  distinct  categories.  First, 
there  was  tho  data  necessary  to  write  the  case  stuay. 

Second,  there  was  additional  data  required  to  test  the 
hypothesis  concerning  validity  of  the  case  as  an  educational 
aid.  Since  the  two  types  of  data  were  significantly  dif¬ 
ferent  with  respect  to  nature  and  source  they  ^.re  di..cu.  „ed 
separately  in  the  following  paragraphs. 

Case  related  data.  Before  specifically  considering 
the  required  data  for  this  area,  and  its  nature  <=ncv  source, 
it  was  first  necessary  to  briefly  outline  tho  completed 
case  study  and  then  consider  tho  parts  of  it  separately. 

In  so  doing,  the  case  outline  in  Appendix  A  was  developed. 
The  remainder  of  this  section  addresses  the  areas  of  the 
outline  and  defines  the  nature  and  sources  of  the  date  that 
were  required  to  complete  the  corresponding  section  of  the 
CO GG  study. 

Tho  introductory  portion  oL  Cc.:  o  t  engirt,  oi 

two  main  parts,  a  statement,  of  background  and  a  statement  of 
desired  learning  objectives  from  the  care.  Data  for  the 
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first  of  these  areas  is  factual  in  nature  and  was  taken 
from  the  background  presented  in  Chapter  1  of  this  paper, 
edited  to  include  those  items  it  yms  consideicci  necessary 
for  the  student. to  know • 

The  statement  of  desired  learning  objectives,  as  the 
j-jqj-j',0  implies,  was  included  in  oider  to  outline  for  the 
student  the  knowledge  he  should  have  on  completion  of  the 
case  study.  Items  in  this  area  were  obtained  through 
discussion  with  Professor  Chsur.ccy  H.  Dean,  Jr.,  of  the 
School  of  Systems  and  hogistics  (AU) ,  Air  Force  Institute 
of  Technology#  in  whose  course  the  care  was  primarily 
intended  to  be  used. 

The  section  of  the  case  entitled  "'flic  B-l  System 
Program  Of  t: ;i «  *•  (SPG)  "  contain:*  three  major  are*.ss.  hhe  j-iisc 
of  these  is  general  in  nature  and  begins  the  "story- line” 
of  the  case  by  introducing  the  characters  involved,  all 
of  which  arc  fictitious.  The*  main  character,  v;ho  is  used 
throughout  the  case,  is  a  young  fir  Force  Captain  (Copt^in 
James  Flake)  who  was  supposedly  newly  assigned  to  the  pro¬ 
curement  section  of  the  B-l  fSFO.  The  purpose  of  the  use  of 
a  story -line  and  characters  v.vr  to  r'd  in  main  to  i  ni  ng 
continuity  in  the  case,  to  gain  the  student’s  interest#  and 
(in  the  case;  of  Captain  Flake)  to  provide  a  character  with 
v/hoin  the  majority  of  the  students  would  bo  »  ole  to  iocitci  . 
The  case  therefore  follows  "Captain  131.  he"  as  he  .learns  the 
rudiments  of  his  new  job.  heg .inning  w.i.ih  his  attendance  <•  t 
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a  SAIPS  briefing#  in  v:hich  essential  information  about 
SAIMS  was  presented.  Data  in  this  section  was  taken  from 
the  available  DoD  and  US/'.r  publications  describing  SAIKS 
and  its  component  parts.  Data  was  selected  from  these 
publications  in  such  detail  as  was  needed  by  the  student 
to  acquire  minimum  acceptable  knowledge  of  SAJMf,  as  defined 
in  the  section  on  desired  learning  outcoi.es.  In  the  second 
part  of  this  section#  the  C/CCS  Criteria  wore  presented  to 
the  student.  Tnis  was  done  in  the  context  of  office  con¬ 
versation  between  Captain  Blake  and  Captain  Bradbury  after 
’the  SAIIIS  briefing.  Data  for  this  portion  of  the  case  was 
taken  from  DoD  Instruction  7000.2#  dated  22  December  1967, 
and  entitled  Performance  Measurement  for  Selected  Acg u  1  s  i  - 
tions.  A  co  mplete  copy  of  the  criteria#  taken  from  this 
publication  was  included  as  an  attachment  to  the  case 
problem  for  the  use  of  those  students  who  wished  to  review 
the  Criteria  in  their  original  form.  The  final  area  covered 
in  this  section  w as  in  the  context  of  conversation  between 
Captain  Blake  and  his  supervisor,  Major  Finley.  The  topics 
were  the  purposes  of  C/SCCC  end  the  method  of  r>  i  ruing  con¬ 
tractor  corr.pli:  nee.  Data  presented  was  extracted  from 
DoD  Instruction  7000.2  (referenced  above  )  and  from  Air 
Force  Regulation  179-7,  dated  27  June  1969#  entitled 
Performance  wc  ;sm>  c  :,cnL _ (JV) _ for  Be  l  r  ote  ■  7.  am’  •  \  1  tions. 

'Hie  next  major  section  of  the  ea  se  si  udy  is 
entitled  "Validation"  and  was  divided  into  three  suk- topics 
for  presentation.  The  first  of  these  is  a  brief  description 
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of  the  validation  process,  as  explained  to  Captain  Blake 
by  a  fellow  worker.  Captain  Bradbury.  The  facts  presented 
were  obtained  from  the  undated  U.3.  Air  Force  Systems  Com¬ 
mand  (SCCCB)  publication  C/SPC3  and  C/DCDC  Validation 
Process  Brief.  As  in  previous  areas,  the  amount  of  detail 
to  include  was  determined  by  review  of  the  learning  objec¬ 
tives  of  the  case  with  regard  to  this  area.  The  second 
sub-topic  in  the  section  presented  the  first  actual  data 
from  the  B-l  SPO  used  in  the  case.  Ibis  was  done  by  having 
Captain  Blake  and  his  co-worker.  Captain  Bradbury,  lcview 
briefly  the  C/SCSC  information  system  description  of  the 
contractor.  Information  presented  was  taken  from  the 
North  American  Aviation  Division  of  North  American  Rockwell, 
as  presented  in  their  publication  Compass:  LAD  Procram 
Planning  end  ? vs  tom  Centro.!  -  The  final  sub -topic  in  the 
“Validation"  section  is  the  Report  of  Ccse/Dchcdtd  e_  C ontrol 
Sys> tern  Demon s t r a t i on ,  dated  December  1970,  written  by  the 
Validation  Team  on  the  North  .American  system  validation 
reviev.%  /.ctual  data  presented  consisted  of  the  C/SCSC 
Demonstration  Review  Checklist,  to  illustrate  the  scope  of 
the  validation  effort,  and  a  single  example  of  supporting 
exhibits  nd  narrative  justification  for  one  .item  on  the 
checklist.  The  j.llu  trative  item  chos-cn,  checklist  item 
l.b.  of  the  organization  cr.it  or  i  j,  vs  s- selected  because  i4- 
is  representative  of  supporting  documental  ion  in  cn neral 

fn  this,  sect 


and  not  parti cv.l  a i  ]y  1  ongl  hy . 


ion  of  the  case 
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as  in  others,  tho  actual  data  v:as  used  primarily  to  lend 
authenticity  to  the  ease  and  to  solicit  interest  from  the 
student  through  association  of  the  case  v:i th  the  B~1  I uinne-d 
Bomber,  a  topic  of  inherent  interest  to  most  Air  Force 
officers  - 

Trie  final  major  topic  area  of  the  case  is  Surveil¬ 
lance.  It  was  included  in  the  content  of  on  orientation 
visit  by  Captain  Blake  to  the  Mr  Force  Plant  Representative 
Office  (APPRO)  serving  the  contractor.  There,  in  the  case 
context,  Mr.  McDonald  of  the  APPRO  Production  Division 
explained  the  surveillance  function-  In  a  subsequent  sub- 
topic  of  this  section,  the  VTork  CreahcovTi  Structure  (KBS) 
was  explained  and  the  interlocking  relationship  v.vs.  illus¬ 
trated.  Finally,  this  section  contained  an  illustration 
of  a  Cost  Performance  Report  and  an  explanation  of  the  data 
thereon.  The  data  presented  was  changed  from  actual  at  the 
request  of  the  B-l  SPO  but  the  form  and  format  were  retained 
from  the  original. 

The  final  two  portions  of  the  case  are  discussion 
questions  on  tho  material  presented  and  suggested  answers  f.or 
the  questions.  The  questions  were  developed  uro:n  the  learning 
objectives  of  the  case  and  the  an;  v  rs  from  :  *  2  ial  pre¬ 

sented  therein.  Kach  is  designed  to  general*  discission 
including  the  main  points  of:  the  related  object  .ivo(s)  .  The 
questions  wore  so  constructed  flint  the  answers  to  each 
could  be  found  in  t  ho  case  or  inlciied  from  infoi  mat  i  on 
pres  on  t cd  1 he re in. 
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Validation  data.  The  data  needed  for  this  area  of 
the  paper  was  used  in  testing  the  hypothesis  presented  in 
Chapter  1  (p-ge  8) ,  that  a  sample  group  of  Air  Force  offi¬ 
cers  v /ho  had  used  the  case  study  would  score  significantly 
higher  on  an  examination  of  the  topic  than  would  a  similar 
group  who  had  had  C/SCSC  presented  to  them  through  a  class¬ 
room  lecture.  This  data  \/u s  collected  by  developing  the 
test  questions  in  Apper.dic  C  and  administering  the  resultant 
examination  to  tv/o  selected  groups  of  officers.  The  first 
group  consisted  of  ten  randomly  Selected  officers  from 
Graduate  Logistics  Class  72A  of  the  Air  Force  Institute  > 
of  Technology,  School  of  Systems  and  Logistics.  Upon  their 
completion  of  the  Financial  Management  Course  (M.S.  5.22), 
which  was  taken  during  their  first  quarter  (8  March  -  14  May 
1971),  the  test  was  administer cd  to  then.  This  group  had 
received  their  instruction  concerning  C/S CSC  through  the 
classroom  lecture  presented  in  the  a  foremen  ttor.ed  course. 

The  second  group  consisted  of  a  group  of  ten  officers 
who  were,  at  that  time,  faculty  members  of  the  School  of 
Systems  and  Logistics.  This  group  worked  through  the  case 
problem  and  was  then  given  the  same  test  as  l he  first  group. 

The  examine t ion  consisted  of  twenty- five  multiple 
choice  questions,  with  a  possible  value  of  four  points  each, 
for  a  total  of  one-hundred  points.  In  scoring  the  results, 
the  full  four  points  were*  given  for  a  corn  f  answer  and  no 
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points  were  given  for  any  other  choice.  Total  score  was 
calculated  for  each  individual  tested  by  multiplying  the 
number  of  correct  answers  by  four.  The  resultant  scores 
were  used  in  the  statistical  test  of  hypothesis  described 
in  Chapter  3. 

Data  Coll ection  Techniques 

To  a  large  extent  data  collection  techniques  were 

discussed  in  the  foregoing  sections  of  this  chapter.  How¬ 
ever,  the  following  points  are  deserving  of  emphasis  or 
s  ur nma  r  i  z  a  t  i  o  n . 

Data  used  in  the  case  study  was  selected  based  on 
the  author's  judgement  of  its  use-fullness  ir.  illustrating 
points  related  to  the  desired  learning  outccr.cs  of  the  case. 
As  such,  it  should  not  be  viewed  by  the  reader  as  compre¬ 
hensive,  nor  should  its  selection  be  consider -id  to  be  ran¬ 
dom.  Since  the  case  was  designed  to  be  a  relatively  short 
introduction  to,  and  overview  of,  the  main  points  of  C/SCSC 
the  data  was  often  edited  to  retain  only  that  which  was 

considered  to  be  essential . 

Data  collected  for  purposes  of  the  statistical,  test 
of  the  usef ullncss  of  the  case,  however,  was  strictly  con¬ 
trolled  to  eliminate  bias  wherever  possible.  Randomness 
of  both  groups  to  be  tested  war,  assured  by  sc  lection  through 
use  of  n  random  number  table.  Potent  ini  groan  n.c.tbc  ’  \.<-ie 

polled  in  advance  of  group  selection  and  el r.  ' noted  from 
consideration  if  they  professed  prior  hnovledce  of  C/SCSC 
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and/or  SAIMS.  Members  of  both  groups  wore  selected  prior 
to  the  time  C/SCSC  was  presented  to  them  and  were  told  in 
advance  they  would  be  ashed  to  tube  a  test.  This  was  con¬ 
sidered  necessary  since  those  taking  the  case  approach 
would  necessarily  have  to  know  the  purpose  in  advance  and 
it  therefore  seemed  that  the  members  of  the  other  group 
should  have  similar  advance  warning  of  the  impending  "test." 

In  addition  to  the  completion  of  the  test  itself, 
the  group  of  faculty  members  were  also  asked  to  provide 
answers  to  several  short  questions  shov:n  in  Appendix  C 
to  this  paper.  These  questions,  four  in  number,  were  used 
by  the  author  to  identify  weak  areas  in  the  case  and  to  gain 
initial  evaluations  of  the  difficulty  and  time  requirement 
of  the'  case.  Answers  to  these  questions  were  considered 
and  evaluated  by  the  author  and,  when  suggestions  see: red 
warranted,  appropriate  changes  were  mace  to  the  case. 


Chapter  3 


ANALYSIS  AND  HYPOTHESIS  TESTING 

The  objective  of  the  statistical  tests  performed 
was  to  be  able  to  make  inferences  as  to  the  possible  validity 
of  the  case  study  developed.  The  lhethod  selected  to  accom¬ 
plish  the  objective  was  to  develop  a  multiple  choice  exami¬ 
nation  covering  the  topics  of  the  case  and  administer  the 
exam  to  two  sample  groups,  one  of  which  had  studied  the 'case. 
Since  the  case  was  developed  for  use. in  the  Financial  Man¬ 
agement  course  at  the  Air  Force  Institute  of  Technology, 
School  of  Systems  and  Logistics,  the  most  useful  test  of 
validity  was  considered  to  be  one  which  would  compare 
examination  results  of  students  of  the  course  to  another 
group.  The  two  sample  groups  were  therefore  chosen  to  be: 

(1)  a  group  of  ten  student  .s  who  had  c om.pl e ted  the  n fore- 
mentioned  course,  and  (2)  an  equal  sized  group  who  had  not 
taken  the  course  but  had  studied  the  case. 

Since  the  course  students  are  primarily  officers  in 
the  U. S.  Air  Force,  the  group  to  study  the  case  was  also 
limited  to  Air  Force  officers,  both  populations  were 
further  defined  to  exclude  officers  professing  prior  knowl¬ 
edge  of  the  subject  matter  of  the  case. 
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Avail  ability  of  prospective  participants  for  inclu- 

*  sion  in  the  croup  to  cover  the  case  was  limited.  The 
decision  v/os  therefore  made  to  draw  members  of  the  "case'1 
group  from  the  faculty  of  the  School  of  Systems  and  Logis¬ 
tics.  This  decision  resulted  in  a  group  composed  primarily 
of  holders  of  Doctoral  degrees.  Consideration  was  there¬ 
fore  given  to  the  possibility  that  comparison  of  results  of 

4  one  group  composed  of  members  holding  Doctoral  degrees 

to  another  croup  composed  of  Masters  degree  candidates 
might  bias  the  test.  After  due  consideration,  it  was 

* 

decided  the  results  would  not  be  significantly  bic.sed  by 
the  educational  difference  of  the  two  groups.  Other  factors 
entering  into  this  decision  were:  (1)  the  Financial 

.  Management  course  devotes  approximately  ten  weeks  to  the 

* 

study  of  Financial  I  hr:  e.  cement  in  the  Federal  Cover  nrivcnt, 

(2)  several  class  meetings  of  the  course  are  directed  to 
topics  related  to  case  materia!,  (3)  the  faculty  gro  ip 
would  be  expected  to  spend  one  to  two  hours  covering  the 
case,  and  (4)  the  stuieni  motivation  to  learn  the  material 

% 

offered  in  the  course  would  be  expected  to  be  greater  than 
the  motivation  of  the  faculty  group,  since  members  of  the 

J,  latter  group  arc  c:e  -  <  moly  busy  an d  have  no  particular 

personal  interest  in  the  outcome  of  the  ex< mu  The  result 
of  considering  all  factors  is  the  conclusion  that  any  excess 
learning  ability  of  the  faculty  group  over  the  student  group 
should  be  sufficiently  offset  by  increased  motivation  of  the 
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student  group.  The  groups  vure  selected  and  the  test 
completed  bar:ed  on  this  ussarr.ption. 

3ej rr Mon  of  the  Statistical, _T cJlt 

The  method  employed  to  test  a  hypothesis  similar 
to  the  one  stated  in  Chapter  1  (page  8)  is  use  of  a  statis¬ 
tical  test  to  determine  whether  or  not  results  of  the  two 
samples  wore  lihely  to  have  come  from,  populations  which  are 
the  reverse  of  the  predicted  difference.  One  of  the  most 
powerful  tests  to  accomplish  this  is  the  t  test,  applied 
to  the  sample  means,  however,  one  of  the  assumptions  neces¬ 
sary  for  this  test  is  normality  of  the  populations  from 
which  the  samples  were  drawn.  Thera  was  no  available 
evidence  to  justify  assumption  of  normality  in  this  test. 
Considerations  of  time  and  the  limited  number  of  faculty 
members  available  to  participate  precluded  drawing  a  suf¬ 
ficiently  large  sample  to  take  advantage  of  the  central 
limit  theorem.  It  was,  therefore,  decided  that  a  non- 
parametric  tent  v;ot:ld  be  appropriate. 

*fho  I  rhi  troy  to  s'  and  Vho  Kolmooorov-Sm.ii  nov 

two  sample  test  both  provide  a  suitable  nonparrmetr l.c  method 
of  making  the  intended  test.  Of  the  two,  the  Kol:  .ogoiov- 
Gmirnov  test  provides  the  higher  power-efficiency  for  small 
samples.  It  was  therefore  chosen  as  llic  test  Vo  be  used. 
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Description  of  the  l^jjuoc^oro)/^ 

Smirnov  Test 

The  Kolmogorov -Smirnov  tv:o  sample  test  determines 
Whether  two  independent  samples  were  drawn  from  populations 
having  the  same  distribution.  'the  one-tailed  version  of 
the  test  may  be  used  to  determine  whether  or  not  the  values 
of  one  of  the  populations  are  larger  than  those  of  the 
other.  Specifically,  the  test  has  been  suggested  as  appro¬ 
priate  for  determining  whether  or  not  scores  of  a  test  gioup 
are  higher  than  those  of  a  control  group. 

Methodology  employed  in  applying  the  Kolmogorov- 

,.2 

Smirnov  test  may  be  summarized  in  four  steps. 

1.  Results  from  the  two  groups  are  arranged  in  a 
cumulative  frequency  distribution,  using  the  same  .intervals 
for  each. 

2.  At  each  point  in  the  above  distribution,  the 
difference  between  the  croups  is  caJcu3atcd  by  subtraction. 

3.  The  largest  difference  in  the  predicted  direc¬ 
tion  (for  a  one-tailed  test)  is  determined  by  inspection. 

4.  For  cample  sizes  less  than  forty— one,  published 
tables  arc  used  to  determine  significance  of  the  cbscrsco 
differences. 


1  f>  i  c  i  n  ey  Siegel,  Krt’-In  11 
]3oha vi  or  nj  r.cicnvyVi,  (Mev:  Yol  >*: 

p.  127. 


ftU  r  i.f  111  'di  \  d  xXAf.  •:  l  for  the 
I  '.oOrnw-I  U  1 J  ,  Jnr.  ,  3  9 bo) 


^ Ibid. ,  p.  1 35 . 


Analysis  of  the  Examination  Results 

The  results  of  the  examination  of  the  two  sample 
groups  are  shown  in  Table  1*  The  scores  displayed  were 
calculated  by  multiplying  the  number  of  correct  responses 
times  four.  Maximum  score  possible  was  or.e -hundred.  The 
methodology  previously  discussed  was  applied  to  the  ordered 
data  shown  in  the  table. 


Table  1 


Ordered  Scores 

of  the  Sample  Groups 

Non-Case 

Case  Study 

Group  Scores 

Group  Scores 

8 

32 

8 

60 

8 

64 

8 

68 

20 

68 

24 

72 

24 

72 

28 

84 

48 

84 

48 

92 

The  statistical  test  was  performed  to  accept  or 
reject  the  null  hypothesis,  which  is:  The  group  of  officers 
who  studied  C/SCSC  without  the  use  of  the  case  study  will 
score  as  high  or  higher  on  the  examination  than  the  group 
who  did  use  the  case- 

The  alternate  hypothesis  is:  The  group  of  officers 
who  studied  C/SCSC  by  use  of  the  case  study  will  score 
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higher  on  the  examination  than  the  group  who  did  not  use 
the  case. 

The  significance  level  set  for  rejection  of  the 
null  hypothesis  is  .05.  Reference  to  Table  L  in  the  Siegel 
text  showed  the  critical  value  to  be  six,  where  is 
the  maximum  difference,  in  the  predicted  direction,  between 
the  two  cumulative  distributions.  Thus,  a  calculated  value 
of  Kd  which  is  equal  to  or  greater  than  six  allows  rejection 
of  the  null  hypothesis,  while  a  calculated  value  less  than 
six  does  not. 

The  cumulative  step  functions  for  the  two  distri¬ 
butions  were  calculated  by  the  method  described  in  Appen¬ 
dix  E.  Results  of  the  calculations  are  shown  in  Table  2. 

The  largest  difference  in  the  predicted  direction  is  nine- 
tenths,  as  indicated  in  the  table.  The  numerator  of  this 
difference  is  taken  as  the  calculated  value  of  There¬ 

fore/  Kq  =  9  is  the  value  to  be  compared  to  the  critical 
Kd  value  (6) .  Since  the  calculated  value  is  larger  than 
the  critical  value,  the  null  hypothesis  is  rejected. 

At  the  .05  significance  level,  the  conclusion  is 
the  sample  group  of  officers  who  did  not  use  the  case  study 
was  not  drawn  from  a  population  scoring  as  high  or  higher 
than  the  population  of  the  sample  group  which  did  use  it. 
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Chapt er  4 


CONCLUSION  AND  RECOriLKDDATiODS 

The  purpose  of  this  research  effort  was  to  develop 
a  case  study  based  on  t ho  Cost/ScLedule  Control  System 
Criteria  (C/SCSC)  as  applied  by  the  13-1  Systems  Program 
Office.  To  that  end  the  case  included  as  Appendix  B  was 
developed.  To  guide  the  endeavor  three  research  questions 
and  one  hypothesis  were  developed.  '‘he  conclusions  in 
this  thesis  are  related  to  the  subjects  therein. 

Research  Quo stion 

Can  the  CosL/Schcciula  Control  Syr: tern  Criteria  be 
presented  in  a  comprehensive,  self-contained  case  study? 

The  case  develop  •  in  i he  course  of  this  research 
effort  does  present  the  Co.-;t/Lchcdulo  Control  System  Cri¬ 
teria  within  the  case  context.  ft  is  comprehensive,  ni 
that  explanations  and  illustrations  of  the  criteria,  related 
systems,  ar.d  major  proves  so;;  involved  in  application  of 
C/SCSC  to  an  acquisition  arc  all  included  in  the  case. 
Finally,  since  the  case  docs  not  r^ly  on  availability  of 
manual regulations,  or  other  publications,  it  is  self- 
contained.  Accordingly,  this  rese-  r  h;  quasi. ion  is  answered 
af  firm.  •  Lively. 
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Research  Question  #2 

Can  such  a  case  study  be  developed  based  on  an 
iictual  C/SCSC  application? 

Throughout  the  case,  topics  explained  are  illustrated 
by  examples  drawn  from  the  B-l  SPO  application  of  C/SCSC  to 
the  north  American  13-1  contract.  The  basic  framework  of  the 
case  is  the  sequence  of  events  followed  by  the  B-l  SPO  in 
establishing  C/SCSC  as  the  basis  of  performance  measure¬ 
ment  on  the  contract.  It  is  therefore  valid  to  conclude 
that  an  application  of  C/SCSC  can  be  used  as  the  basis  of  a 
case  study. 

Research  Question 

Could  such  a  case  study  be  useful  in.  imparting  basic 
understanding  of  the  C/SCCC  system,  methods,  and  procedures? 

To  facilitate  evaluation  of  the  extent  to  which  the 
case  study  imparts  uncle rs landing  of  C/HCSC  an  examination 
v/as  developed  and  administered.  The  scox~es  of  tv.v.  inde¬ 
pendent  samples  of  Air  Force  officers  were  analyzed  to  test 
a  hypothesis  related  to  the  research  question.  The  hypothe¬ 
sis  tested  was :  A  sample  group  of  officers  v.ho  had  used 
the  case  study  would  score  significantly  higher  on  an 
examination  consisting  of  questions  pertaining  to  C/SCSC 
and  B.Mj'S  than  would  a  similar  group  who  had  the  subject 
presented  to  them  through  a  classroom  lectin  o. 

'iVo  sample  groups  were  chose  n  and  ih  oiamir.al  ion 
in  Appendix  C  wax  udmi ni si  oreci  to  ner  tm  :■  of  e  ach.  'ihe 
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results  of  the  examination,  significant  at  the  .05  level, 
support  the  hypothesis  stated  above.  The  case  is  therefore 
useful  education:1 1  vehicle  in  this  regard. 


accepted  as  a 


Re  com;  no  ndat  ions 

Based  on  the  conclusions  of  the  research  effort  it 
is  recommended  the  case  study  in  Appendix  B  be  incorporated 
into  the  Financial  Management  course  at  the  School  of  Sys¬ 
tems  and  Logistics.  Air  Force  Institute  of  Technology. 

Also  recommended  is  repetition  cf  the  statistical  validity 
test  of  the  case  subsequent  to  the  first  offering  in  which 
the  case,  is  used.  The  examination  in  Appendix  B . should  be 
administered  to  students  of  the  class  and  their  scores 
used  to  construct  a  cumulative  step  function,  as  explained 
in  Appendix  D.  The  results  should  be  compared  to  those 
of  the  Financial  Management  students  of  Graduate  Logistics 
Class  72A,  who  comprise  the  non-case  study  group  in  Chap¬ 
ter  3.  Further  use  of  the  case  should  consider  the  results 
of  tests  between  the  two  student  croups  as  well  as  the 
instructor's  evaluation  of  the  level  of  understanding 
achieved  by  the  tv;o  groups. 
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THE  B-l  WANNED  BOMBER  CASE 


Introduction 

This  case  has  been  developed  specifically  for  use 
in  the  Financial  Management  course  (M.S.  5.22)  at  the 
School  of  Systems  and  logistics*  .It  is  intended  as  an 
introductory  overviev;  of  the  Departmant  of  Defense  (DoD) 
Cos t/ Schedule  Control  System  Criteria  (C/SCSC)  and  the 
environment  v/ithin  which  it  is  used.  References  to  the 
B-l  System  Program  Office  (SBO)  and  to  the  data  from  the 
North  American  Rockwell  contract  are  for  illustrative 
purposes  only  and  should  not  be  taken  as  factual  repre¬ 
sentations  of  the  state  of  the  contract.  Much  of  the  data 
presented  has  been  charged  to  make  it  .suitable  for  illus¬ 
trative  purposes  but  )>  sic  form  myj  format  has  been 
retained. 

Background 

During  the  years  from  the  end  of  World  War  JJ  to 
the  m id-1960:;  rapid  technological  change  and  the  cold  war 
requirement  for  a  standing  military  fore?  comoin^d  to 
produce  a  rapidly  expanding  deft.  i. so  industry-.  I  air  nor  and 
more  complex  weapon  s./support  systems  costing  millions  of 
dollars  contributed  to  a  rapid  expansion  of  the  defense 
budget.  As  DoD  cornpe*  •  1  for  large)'  rhai  is  of  the.  Federal 
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tax  dollar  both  congressional  and  public  pressure  increased 
demands  for  better  management  of  the  defense  dollar. 

In  response  to  demands  for  better  management  and 
increased  visibility  into  the  uses  of  tax  dollars  on  large 
defense  contracts,  numerous  management  information  systems 
v/ere  developed  and  imposed  on  contractors.  Kany  of  these 
systems  met  some  degree  of  success  but,  even  so,  disastrous 
cost  overruns  and  schedule  slippages  continued  to  occur 
with  embarrassing  regularity.  Further,  the  major  defense 
industry  contractors  began  to  complain  that  Pol)  v?oS  pre¬ 
empting  their  management  prerogatives  by  imposing  detailed 
and  overlapping  systems  on  them. 

Xn  response  the  Air  Force  Systems  Command  (7.1  SC) 
in  1962,  adopted  the  Verb  breakdown  Structure  (WBS)  concept 
developed  by  the  navy  ir.  19oB  for  the  „  r  Polaris  missile 
project.  7-.F3C  augments -j  •'*  •  w.i  th  a  concept  whereby 
r»*  ntractor  would  he  uliovrca  to  develop  his  own  management 
./stem,  designed  around  the  v;ns,  which  v:ould  have  to  act 
certain  specified  minimum  criteria  of  acceptability. 

Based  on  dr  .one t:  rated  success  of  the  not:  system  7-.FSC 
published  its  Cost/ocher. ule  '  3 aiming  «  no  Control  .Specif  i~ 
cation  (C/JPJ3)  in  June  1966. 

Tn  August  1966,  the  then  rociotnjy  of  Defense, 
Robert  3.  DcITan  ra,  launch? d  t  ho  Dob  Resource  Panaot ment 

K> 

3ys terns  (IC’.S)  with  the  issu  ::cc  of  Dob  Directive  7000.1. 

PJ*3  vms  an  cittc:  pt  to  ir  prove  financial  m.  m jt  V  rouch - 

out  all  of  resource  acquisition,  ut  1 1  turn,  and 
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retirement.  Implementation  in  the  acquisition  phase  was 
begun  in  December  1967  with  the  issuance  of  DoO  Instruc¬ 
tion  7000.2  entitled  "Cost/Schedule  Control  System  Cri¬ 
teria"  (C/SCSC)  .  This  instruction  adopted  7.rSC*s  C/SPCS  for 
DoD  wide  use.  It  is  this  system  that  is  discussed  in  the 
remainder  of  this  case. 


Lea  mi  nr j  Objectives 

C/SCSC  and  the  Selected  Acquisition  Information 
Management  System  (SAMS),  of  which  it  is  a  portion,  ere 
complex  interlocking  financial  management  systems  designed 
to  provide  the  DoD  manager  with  the  tools  with  which  he  can 
manage.  To  do  so  they  provide  for  furnishing  needed  infor¬ 
mation  to  managers  at  all  levels  of  the  Federal  Government. 
In  many  instances,  they  arc  necessarily  detailed  and  a 
thorough  working  knowledge  of  them  c- res  only  with  months 
or  years  of  experience.  This  case  and  the  course  in  v.hicn 
it  is  used  c  a  not  possibly  achieve  tna.t  level  or  detail  in 
the  time  available.  The  intent  is,  therefore,  to  provide 
the  student  an  overview  or  framework  on  vhicn  t  '  build  in  >h 
future.  Accordingly,  the  following  objectives  have  been 
developed  lo  aid  the  student  in  rt  -agnizing  the  desired 
level  of  knowledge  and  und  :  .banding. 

Upon  completion  of  v.liir.  <-•.>;.<  ih-  stwernt  should. 

1.  Know  what  the  Selected  Acgui r.iUmi  Inf ons  lien 
Manor;,. ment.  System  (MAIMS)  is,  know  1.  w  it  vs  onjuiii  ;.<  d, 
iir.d  recognise  .its  posit, ie-vi  as  a  pal  t  <  f  It  s  . 
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2.  Understand  the  need  and  reasons  for  SAMS 
existence. 

3.  Know  the  four  major  sub- systems  of  SAB'S  end 
understand  the  purpose  of  each. 

4.  Know  the  five  areas  of  the  Cost/Schedule 
Control  System  Criteria  and  understand  the  specifications 
made  by  each. 

5.  Understand  the  Work  Breakdown  Structure  con¬ 
cept,  the  earned  value  concept,  and  the  reasons  for  their 
use. 

6.  Know  the  major  events  and  chronological  sequence 
of  their  occurrences  in  a  C/SCSC  application. 

7.  Know  the  functions  served  by  Validation  and 
Surveillance  and  understand  the  reasons  for  them. 

8.  Understand  bob/  I r  Force  policy  concerning 
application  of  C/SCSC. 

The  >3-3  System  Program  Office 

Captain  James  Blake  had  reported  to  the  SPO  only 
a  few  days  before.  As  a  procurement  officer  in  the  Air 
Force  Logistics  Command,  he  had  heard  of  SAM'S  and  C/SCSC 
but  had  only  a  brief  acquaint ancc:  with  either.  He  was, 
therefore,  somewhat  uneasy  upon  learning  he  would  be  very 
much  .involved  with  both  in  his  new  position  or  one  of  t3ie 
contracting  officers  for  the  SPO.  He  was.  pleased  when  3ms 
new  supervisor.  Major  Finley,  told  him  to  attend  briefing 


by  captain  Bradbury  of  the  SPO  Program  Evaluation  D5-. 
Ihe  Briefing  was  an  informal  one  v;hich  Captain  Bradb- 
often  gave  for  new  officers  at  the  S*0. 

The  S7-.IMS  Brie  ring. 

7:3  the  briefing  began.  Contain  Blake  took  r.o- 
the  significant  points  covered.  Captain  Bramury  m 
duccd  his  presentation  with  a  shout  history'  o-  tr.e. 
nont  of  the  Resource  Management  System  (k-S)  ry  Xk>^. 
then  explained  how  RMS  is  divided  into  four  croup-  o 
terns’  each  with  a  different  area  of  concern.  S/,I 
continued,  was  a  part  of  the  Acquisition  Infer,  -tier 
Management  System  (AIMS)  portion  of  HVS.  To  illur  - 
he  showed  a  chart  such  as  the  one  in  Figure  1.  cn  I- 
The  chert  showed  AIMS  to  bo  one  Oi.  - v*°  s-v  ■ 
rcanuejeraent  of  ncuatn.  -0}  ‘*rj  ■  1  G  '"1;  ( 

tory  Management  oyc  ter.  von  101  1#,  -c  r"'*’  r  ~  7 

in  the  DoD  inventor ion  nd  won  ror  Uv— t  Ltli;  * 

acquisition  phene.  Since  Information  iupv+z  to  A 
note  with  the  contractors  rather  than  within 
systems  arc  required  to  control  the  tyjxi  ^ -  -  ch 
the  data*  liven  system,  he  raj  a,  v;cl'J 
s'tion  TrT  on: a '-ion  T'anr  qentnt  hy  (.AT*—##  r 

acquisitions,  and  a  collection  o*.  h. 

Capital  Acquisitions  for  mailer  procure.  .  c  * 

nracihury  next  atldrnnra  *1  u,i'J 

are  a  part  ol  it.  lie  explained  th  t  !  h'  r 
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information  is  used  to  build  a  data  bise  access  able  to  pro¬ 
curement  offices  throughout  Dot).  Information  from  the  file 
is  used  to  support  cost  estimating,  programing,  and  bud¬ 
geting  for  procure::, ent  of  future  systems. 

Contract  Fund  Status  Reoortr:  ( C F5 R )  .  The  CFSR  ex- 
tracts  data  from  the  contractor's  i  r.  forma  t  ion  system  red  a— 
tivo  to  funding  requirements  of  the  contract.  Actual  costs 
the  contractor  has  incurred  in  performing  contract  work  and 
applied  to  the  contract  cost  accounts  a ic  reported,  as  are 
projected  costs  to  be  incurred  and  applio  in  the  future. 
The  cdsts  arc  tine  phased  in  the  report  to  indicate  which 
fiscal  quarter  tho  contractor  aspects  to  null  the  Govern¬ 
ment  •  Information  on  the?  report  if:  used  by  *  no  by stems 
Program  Office  to  update  foreran'*  fund  require:'  ants ,  to  aid 
in  planning  and  deciric;.  :  V.  i  r.g  <  n  cIkiScv  /  affect ir.g  the 
requi remenl  for  fund/,  to  envelop  fund  require  ents 

and  budget  estimator  to  support  approved  v/egrar.s. 
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assuring  that  data  on  the  contractor's  EI3,  CIR,  ar-d  CFSR 
reports  are  valid  and  compatible  v.'ith  internal  DoS  lrfor- 
nation  systems.  it  also  provide;;  Use  basis  by  which  the 
Air  Force  can  measure  the  contractor's  progress  in  meeting 
the  contract  v/orh  schedule  and  provides  a  method  to  ideiitrfy 
cost  overruns  and  underruns  on  the  wor);  being  done.  The 
technical  sub-- system,  SEM,  requires  the  contractor  to  inter¬ 
face  Uie  technical  performance  aspects  of  the;  v.'ctrcn  or  sup¬ 
port  system  with  .the  C/SC5C.  It  also  provider  the  basis  for 
validating  Iris  own  System  Engineering  K  ana  germ  rt  program. 

The  Cost  /3  cheap  l «: ;  ( * o p.  1  ro  1  5v : •  1:  e:n 

enter: a  ( d/ ; ; ■  '3  J j 

After  the  r./MXE  brief i no  ended,  Captri:.  MU:o  a:  ‘ad 
Captain  Rradbury  to  explain  U.  c/::crc  in  non  rf  il.  In 
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Organisation  criteria.  Criteria  .in  this  area  set 
minimum  standards  for  the  contract  ox  to  meet  in  organizing 
to  perform  the  work  required  by  the  contract.  To  comply, 
the  contractor  must:  (1)  define  all  work  to  done  to  sup¬ 
port  the  contract,  (2)  identify  the  sub- cent  rector  cr  ele¬ 
ment  in  his  ov;n  organization  that  will  be  responsible  for 
the  work,  and  (3)  provide  for  integration  of  the  work  into 
his  organizational  structure.  The  contractor's  system  is 
also  required  to  provide  for  reliable  perf or;  ~r.ee  measure¬ 
ment  once  the  contract  work  begins. 


P 1  a nninn  and  lb idoot  i  n< :  cr.fi  rrt:.  Tr s  re  c r 1 1 <  y i 
require  U.e  contractor  to  ctpar.Uc.ly  idhouule  atd  toudyct  for 
the  elements  of  work  identified  und<  r  the  Organs  :*a  c  i  on  cri-. 
tcria.  They  also  requir*  th  *  any  contingency  f —  xnowTi 


as  inane  gome  nt  rtTMVe:;— i  •  .  m  s  t  i :  .3  <  *> ;  and  as.  nr/  d  te  ape 
cdftc  ri  -  r.acor i  a l  rxx*it  _or.  for  ^’oni.rol  ■  A.not  r/  r  i ; pro¬ 
vision  requires  the  total  of  budge. ir*d  work  «  r.  i  management 
reserves  be  used  to  cstanlioh  a  buoset  ha  sc ) :  ra  for  the 
contract.  This  baseline  is  used  1  ter  as  thr  Unr  against 


v/1 1 i ch  i x» r f o n rm nc e  i  s  no?  sered . 


Accoun  *  i  iv :  c  r  i  t r  >*  i  *  .  Vh*  sr  *  c/'  i  t  *  v  •  *  r  t '-*1  >1  i  •  n  r.i  ni  — 
mum  standards  for  the  contractor  '  j  *»  rs:  :  ysf r  .  One 

major  requirement  prohibits  applie  lion  of  cost:;  to  ’  con¬ 
tract  cost  accounts  prior  to  11k  nr-ounHr  :  r^i.iod  m  vhi^h 
v/ork  is  performed.  Another  r^quirr:  all  cos*  incurred — 

lx>th  direct  and  indirect--to  be  applied  t  r  1  *e.  *}  p:  o:  r.i at  e 
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cost  accounts.  Finally,  the  contractor's  system  is  required 
to  provide  unit  costs  for  completed  units  and  provide  a 
traceable  audit  trail  that  can  be  used  to  verify  those  costs. 

Analysis  criteria.  This  area  consists  of  criteria 
establishing  nanager.'.er.t  analysis  standards  fer  the  contrac¬ 
tor's  information  and  management  systems.  fIhey  still 

referred  to  cs  "reporting"  criteria  in  the  DoD  Instruction 
but  are  most  commonly  known  as  "analysis"  crie*  t  ie.  ^he 
basis  for  analysis  is  provided  by  requiring  the-  contractor's 
system  to  provide  monthly  totals  of  costs  in  three^ categor¬ 
ies.  First/ total'  budgeted  costs  for  work  scheduled  to  be 
completed  must  be  available  for  each  work  element  -or  pack-  - 
age.  Next,  total  budget*  :  costs  fox  v:om  actually  per  x.e>i;r.eb 
on  each  v:ork  package  must  be  v';  J.m1<  .  *  in  blv,  octvi«-3 

costs  .incurred  in  per:? ox  *  t :  r  ccajletcd  v:ork  must  »>c 
collected  and  totaled  f. or  #  *n  v,*c  n*.  paeftaq*  •  ino  conticf  i,wr 
is  then  rcciu i red  to  ts-Iysc  vari*  i  s  c*nlcul«-t*  j  from  t  net  e 
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(80%  x  $10, 000) .  The  variance  of  $2,000  in  negative,  indi¬ 
ting  it  is  unfavorable.  Similarly,  if  cost  accounts  showed 
the  v/ork  actually  performs i  cost  $8,500,  the  contractor  would 
calculate  a  cost  variance.  In  this  case,  the  variance  would 
be  the  BCWP,  which  the  budget  indicated  the  work  should  have 
cost,  minus  the  Actual  Cost  of  Work  Performed  (ACWP) ,  which 
is  whet  it  did  cost.  The  difference  is  -$500  ($8,000  - 
$8,500)  and  is  again  negative  and  tin  ref ore  unfavorable. 
Similar  variances  would  be  calculated  for  each  element  of 
work  that  had  a  separate  budget  established  and  the  contrac¬ 
tor  would  be  required  to  identify  the  reasons  for  the  vari¬ 
ances  and  cither  explain  them  or,  if  appropri;  to, *  initiate 
action  to  correct  t  v  rrc-bl  •  that  c.  n;u  d  them. 
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chanc:  1  wo rk  requirement*  This  is  necessary  to  insure  con¬ 
tinuity  of  the  budget  baseline  is  not  lost  due  to  inability 
to  trace  the  effects  of  changes.  The  contractor  is  prohib¬ 
ited  from  using  previously  budge  red  amounts  to  pc  i  form  work 
resulting  from  the  change  cr  new  budgets  to  cover  previously 
authorized  work.  Retroactive  changes  to  cost  accounts  are 
prohibited  to  avoid  performance  of  work  in  anticipation  of 
a  contract  change.  Otherwise,  contractors  might  request 
changes  and  then  proceed  with  the  work.  Then,  if  bU  Govern¬ 
ment  refuses  the  request,  it  is  too  late* because  the  work  is 
already  done.  The  contractor  m-  y  rot  be  paid  for  the  unau¬ 
thorized  work  but  might:  have  to  undo  its  effect:  before  pro¬ 
ceeding,  thus  causing  a  del. y  in  work  completion.  The  final 
requi  rement  of  the  ciifcr i ...  is  for  the  contractor  to  project 
the  effects  of  c  nanc.es  «*  r.e  rcr ^  v  ri  a  ness  to  civ*  .*  revised 
estimate  of  the  total  corf;  act  b;e  at  cor  pi  otic?  of  all 
v/ork.  These  projectors:  rust  l  reconciled  with  t  h*  budget, 
the  contract  trim,  <.nu  th-  fur  ring  r<qi:i  ro:vr.t;:  shown  on 
the  Contract  Tunas  Gtatus  Repea  t  for  the  period. 

Ob  fee fi vos  o  the  f>ys  *  r~.  * 

V.lion  Mpfain  Pradbvry  finished  his  esrd  **n  i  on  of 
the  Cos  t/  Gchcd  ulo  Control  Cystr  r  Cri  t  c  i  ■»  t  Contain  M  .ke 
returned  to  his  own  office.  lie  Celt  he  had  !<■  arsed  a  rrront 
deal  ab'out  C/”;-:c  but.  s till  did  r.cl  fully  understand  the 
reasons  for  its  existence  or  ju  *  how  contractor  c<  pjjanco 
was  enforced.  He  therefore  ask'  ,  Major  in  by  to  explain 
aspoct.s  of  the  ;.ystrn. 


those 


pi  ^  f, 
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Major  Finley  began  by  explaining  that  the  primary 
purpose  of  the  Cost/Schedule  Control  System  is  to  provide 
a  method  acceptable  to  both  industry  and  the  Government 
\/hereby  an  acceptable  basis  for  performance  measurement  car. 
be  established.  The  criteria  are  not  an  end  in  themselves 
but  simply  a  method  by  vhich  both  parties  can  be  assured 
that  information  from  the  contractor's  management  control 
system  will  bo  sufficient  to  permit  managers  on  both  sides 
to  perform  their  jobs.  Neither  the  Govt*rn.’!.C‘p.t  nor  industry 
believes  it  i:-:  appropriate — or  ever,  i  can  bio. — for  :  tanuord 

system  t:o  be  developed  f  vreed  on  r:l  1  cor.t  tact. or;  because 

internal  conditions  and  rc  qui  ret  nts  are  varied.  'Hie  Govern, 
inert,  however,  must  be  assured  thot  contractor  * s  system 
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to  the  firm  thut  meets  all  requirement.';  at  the  lowest  price. 
The  contract  incorporates  the  requirements  of  the  original 
RFP  and  thereby  contractually  re-quires  the  contractor  to 
comply  with  the  criteria.  The  /-dr  Force  then  reviews  the 
contractor's  management  information  system  in  detail  and. 
if  ’t  meets  the  criteria  in  all  respects,  validates  it. 

One  contract  provision  requires  the  contractor  to  include 
C/SCSC  requirements  in  major  subcontracts  identified  by  v.he 
SPO  and  contractor  as  critical  to  the  ovciall  program,  Sub 
contractors  ore  then  required  by  their  contract  with  the 
prime  to  design  and  implement  an  acceptable  system,  unless 
they  have  previously  done  so.  Government  personnel  olso 
review  and  validate  the  sub-contractors'  systems,  since 
competitive  considerations  usually  prohibit  allowing  the  t  rime 
contractor  access  to  internal  records. 

At  this  point  Captain  Birhc  interrupted  and  ashed 
Major  Finley  how  and  why  the  Air  Force  validates  the  con¬ 
tractor's  information  system.  Major  Finley  suggested  that 
he  postpone  the  explanation  until  the  next  day  and  arrange 
to  have  Captain  Bradbury  explain  it,  since  he  had  been  on 
the  Validation  team  for  north  American. 

Valiant  ion 


The  next  mox nine  when  Captain  Blake  came  in,  he. 
went  straight  to  Captain  Bradbury 'r  olfice.  Captain  Brna- 
bury  was  waiting  lor  him  with  an  array  of  documents  and 
papers  concerning  the  validation  pro.  o:  a,  the  i’orl.h  /meriean 
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information  system  (known  an  "COMPASS”)  ,  and  results  of 
the  validation  demonstration  accomplished  after  the  contract 
was  awarded.  As  the  two  men  reviewed  the  documents.  Captain 
Bradbury  explained  the  major  points  of  each. 

rrhe  APSC  Validation  Brief 

The  first  document  was  an  Air  Force  Systems  Command 
Brief  on  the  C/SCSC  Validation  process.  Captain  Bradbury 
pointed  out  that  the  Brief  was  the  major  guide  used  in 
validating  a  contractor's  system.  It  began  with  a  state¬ 
ment  of  purpose  for  the  validation  process.  Assurance  that 
the  contractor’s  system  would  comply  with  the  criteria 
was  one  of  several  objectives.  Others  v;ero:  (1)  insurance 
of  an  effective  operating  cost/schcdule  control  system  by 
the  contractor,  (2)  development  of  understanding  of  the 
contractor's  system  by  government  personnel,  and  (3)  develop¬ 
ment  of  a  trained  government  staff  v;ho  would  be  fainilieir 
with  contractor  management  operations  and  the  use  of  perform¬ 
ance  measurement  data. 

The  Brief  also  included  a  milestone  list  of  the 
events  leading  to  validation  of  the  contractor's  system 
and  an  explanation  of  the  process.  There  v:orc  eleven  mile¬ 
stones,  covering  a  period  of  nearly  three -hundred  days  (see 
Figure  2,  on  page  51. 

The  validation  process  begins  before  contract.  av:nrd 
when  a  pre-award  system  design  review  is  held  for  prospcc- 
tivo  contractors.  7\  t  this  mooting  they  a  re  brio  Cod  on  the 
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Figure  2* 

Milestone::  in  tho  V.,J  i,;.„  io„  r,„ 
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* 


to  valido'tion^t'^the^contractor^s^yctern.  °Vonts  ^ding 


1.  DoD  Program  and  Contractors 

fS1S/^Gd  for  application  of 
M  C/S PCS  or  DoD  C/SCSC 

2.  Pre-award  management  control 

system  design  review 

3. 

4. 


Contract  award. 

^ tart  preliminary'  validation 

nesting  (pre-demo) . 


5.  start  validation  testing  (dcrr.o) . 

6.  Complete  validation  testing. 

7.  Report  written  and  placed  in 

coordination 

S.  Report  submitted  to  contractor 

9.  Fix-it  conference  (plan  and 
schedule  discussion) . 

10'  R1^trao?n  anB/°e  intenance 
^nd  ourvcillnnce  Program 

11*  Fi n  a 1  Rcpor t 


Approx.  Ho.  of 
Calendar  Days 
Between  Phases 


(Variable) 

(Variable) 

(Variable) 

90 

15 

30 

30 

30 

60 

10-30 

10 


I 


Cz  ^?j 


52 

requirements  their  information  systems  must  meet  to  1x2 
acceptable.  Questions  ere  a  ns  we red  end  every  attempt  is 
made  to  assure  that  prospective  contractor's  understand 
what  is  required.  After  proposals  are  received  and  the 
contract  is  av/a rded,  che  successful  contractor  '  s  syste  .ls 
reviewed  in  more  detail  hy  the  STO  and  obvious  def ieieneies 
are  pointed  out  so  they  may  be  corrected  before  the  Vali¬ 
dation  review.  Validation  testing  then  begins.  The  demon¬ 
stration  team,  consisting  of  about  twenty  members  from,  the 
SPO,  the  Air  Force  Plant  he pr e a e n t a t i v o  Office  (7.FPR0)  ,  and 
the  Defense  Contract  Audit  Agency  (lOC-AA) ;  travels  to  the 
contractor's  facility  ar.d  spends  thirty  days  observing  the 
management  control  and  information  system  in  operation, 
vrhen  this  period  is  complete  the  team  withdraws  from  the 
contractor’s  facility  to  write  the  demonstration  review1 
report.  After  the  report  is  written  and  coordinated  with 
the  SPO  Director,  the  report  is  final  end  constitutes  vali¬ 
dation  of  his  system.  C  them  raise  a  fix-it  conference  is  held 
to  explain  deficiencies.  The  contractor  is  given  time  to 
me  he  necessary  changes  and  a  red  ew.on  strati,  on  is  held .  In 
the  redomons  trot  ion,  changed  portion;;  of  the  system  arc 
observed  to  determine?  if  thry  moot  the  criteria:.  This  pro¬ 
cess  is  repeated  until  all  criteria  are  net.  The:  final 
report  is  then  issued  valid  ting  the  system. 

As  Captain  \v*  dbury  completed  the  7-.FPP  brief  lie 
commented  that  the  ho:  th  A:.:  i  ic:m.  (!:/,)  .*  ysd  e;n,  called 
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“COMPASS,  “  v/o.s  validated  on  the  iirr.t  try.  He  also  said 
one  o£  the  pamphlets  contained  a  description  of  the  HA  sys¬ 
tem. 


4 


COMPASS 

'Hie  pamphlet  on  COMIV-.MS  was  published  by  Hort.1i 
American  to  explain  the  program  planning  and  control  system 
used  on  defense  contracts.  The  system  was  summarised  under 
three  headings:  Planning,  Authorizing,  and  Controlling. 

Tne  first  of  tliese.  Planning,  tnl.es  place  primarily 
before  contract  award  and  consists  or  br easing  the  work 
down  into  individual  parts  and  identifying  who  is  to  do  what 
and  when  and  how  it  is  to  bn  done.  Tnis  is  begun  by  exam¬ 
ining  the  schedule  of  items  and  work  statement  in  the 
Reel  nos  t  for  Proposals.  bacli  item  is  d.Lvicoa  .into  tine  P-  ^  ^ 
ucts  ar.d  services  necessary  to  produce  it.  these  1 :  tken 
subdivided  into  work  pack;  ges  which  result  in  a  fork  Break¬ 
down  Structure  (V.’BS)  res-  ..bl  ir.g  an  organize 1 3 on  chart  (see 
Attachment  -2).  Use  of  the  KOS,  Captain  Bradbury  explained, 
is  required  by  the  Organization  criteria.  The  Government 
provides  the  top  three  levels  in  Jill—  .CTD-SSI  .  Possible 
Level  1  elements  are  Aircraft,  Licet  ronic:  ,  I'.is.'.ilo,  Ord- 


;yn  lo:..; .  The 
*  "A.i  rvroft  c  ::i. 

T  i-STJ.i.  'Jho  cc:v 


nancc,  .Ship,  Space,  nnd  Surface  Vehicle 
hovel  1  YJBS  element  on  l ho  KA  coni  <‘i.  -i 
Lovelr;  2  nnd  3  are  alro  provided  in  th 
tractor  mu:;  t  continue  Iho  hrcuhdc.v:);  to  l  hr  work  pnrkacic 
love].  Loch  ^pt.ry  on  t  h*  V.’LS  ;ir;  then  .union' d  to  ;  n  orcjen  i- 
y.u  t  Aon.“i3  cj  <  n.cnt  that  i  )  t  ho  n  "j  i  s.iblt*  J  eu  i  t:» 
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accompl i shment .  Xhis  enables  responsible  imincigors  to  plan 

1 

-  . 

how  and  v/hc-n  their  work  packages  'will  be  performed,  subject 

$ 

to  control  of  the  program  manager. 

. 

£ 

| 

Once  the  contract  is  awarded  work  may  commence. 

i 

Work  authorizations  are  issued  to  f- notional  managers  at 
the  appropriate  times  to  assure  timely  completion  of  work 

i 

c 

• 

packages.  Controlling,  the  third  phase,  then  begins.  On 
a  monthly  basis,  each  manager  reports  the  status  of  bis 

. 

J 

work  packages.  These  status  reports  are  used  by* the  Program 
Manager  to  calculate  and  report  variances  in  cost  or  sched¬ 

k 

ule.  ■  The  .resorts  co  both  to  the  managers  responsible,  and, 

in  summarized  form,  to  the  Government. 

* 


k 

Th^  north  Arr.eri  rvn  Dc-raon  s  t r a  t  i  on 

Report 

The  last  document  Captain  Bradbury  examined  was  the 

demonstration  report  fro:;*  the  Itorth  American  validation. 

TTie  renort  v«c*3  in  three  sections,  the  f  irst  of  v.’hich  v.t.s 

Findings  and  Kecorfincndati  ons .  This  part  v:as  a  fifty  page 

summary  of  the  t emn  findings*  Based  on  their  findings#  the 

* 

validation  teem  rccorcacndcu  that  the  contractor’s  system 

be  validated*  Vtith  the  a}  provcl  rignutmes  cf fined,  th.is 

* 

V 

4 

section  of  the  renort  constituted  the  official  final  repox t 

sent  to  the  contractor  as  evidence  of  validation.  An  long 

as  the  contractor  maintains  the  information  system  as  aos- 

u 

cri bed  in  the  renvort  it  nor-d  not  to  row 1  i d«,tod —  even  ror  a 

future  contract  requiring  C/ilCSC. 

i 
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Section  Ii  of  the  report --some  270  paces  long- -con¬ 
sisted  of  the  validation  checklist  audit:  and  supporting 
exhibits.  Captain  Bradbury  pointed  out  that  the  validation 
team  had  used  the  Air  Force  Systems  Command’s  ‘tbcmonstration 
Revie\/  Checklist  for  C/JC3C "  in  conducting  the  review  (see 
Attachment  ~3)  .  The  checklist  contains  entries  ror  each 
of  the  criteria  and  provides  space  for  team  members  to 
indicate  whether  or  not  the  contractor’s  system  complies.  , 

<ftlQ  remarks  column  is  used  to  reference  supporting  exhibits 
for  each  item.  As  an  example  of  thcr  supporting .documenta¬ 
tion,  Captain  Bradbury  cited  paragreph  l.b.  of  the  organi¬ 
zation  criteria,  on  page  .1  of  the  checklist.  Exhibit  1-1, 
cited  opposite  that  item  of  the  checklist,  and  the  accom¬ 
panying  verbal  comments  (Attachment  3,  pages  5  and  10}  are 
typical  of  the  documents.  Lion  for  most  checklist  i^cv.s. 
lie  also  c i  ted  i  terns  1 .  c . ,  2  -  a .  ,  2 . b.  ,  3 .  d . ,  4 . a .  ,  <~nd  6 .  c . 
of  the  organization  criteria  cm  entries  validated  by  detailed 
reviews  of  the  contractor’s  system  description  documents  and, 
where  appropriate,  by  verifying  the  existence  cm  internal 
procedures  to  assure  compliance  with  the  critcn  .  Other 
checklist  items,  such  as  l.a.,  l.b.,  !.c.,  l.c.,  l.e.,  an *3 
2. a  of  the  accounting  criteria  arc  validated  by  beginning 
with  source  documents  and  tracing  them  thxovgl  l  h«  system 
to  verify  appropriate  processing.  Copjics  or  *  hr  comments 
and  detailed  write-ups  of  the  processing  on  included  as 
supporting  exhibits  to  justify  the  conclusion:  and  recom¬ 
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Section  III  of  the  validation  report  contains  copies 
of  the  source  documents  reviewed  and  the  original  worksheets 
used  by  team  members  during  the  validation  review.  Tt  is 
retained  by  thie  SPO  Performance  Measurement  Division  as 
detailed  documentation  for  the  first  two  sections. 

At  this  point  all  of  the  documents  had  been  reviewed. 
The  discussion  ended  and  Captain  Fluke  started  back  to  his 
office.  He  was  beginning  to  understand  the  criteria  and 
the  process  that  occurred  in  applying  them  to  a  program. 
However,  he  still  wanted  to  review  some  of  the  reports  and 
have  them  explained.  Major  Finley  later  suggested  post¬ 
poning  review  of  the  reports  until  the  orientation  trip  to 
the  AFPRO  the  following  week.  There  he  would  not  only  see 
the  latest  reports,  but  also  observe  the  7.FFR0  surveillance 
activities. 

Curved  Hance 


A  few  days  later  Captain  Flake  arrived  at  the  Forth 
American  (HA)  plant  at  bos  Angeles  International  Airport, 
the  home  of  the  Los  /.ngclos  Division.  He  was  met  by 
Mr.  McDonald  of  the  7.FPRO  Production  Division,  who  took,  him 
to  the  AFPRO  offices.  Hr.  McDonald  was  one  of  several  pro¬ 
duction  men  who  worked  primarily  on  the  Hi  contract  and  had 
Dec *n  asked  to  explain  he  AFPRO  surveillance  progr^:  to 
Captain  Pinko. 
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11)0  Survoi  llnn^e  Program 

Mr.  McDonald  began  by  explaining  the  surveillance 
program  had  been  provided  for  in  the  original  DoD  Instruc¬ 
tion  establishing  the  performance  measurement  system,  of 
which  C/SCSC  v.us  a  part.  According  to  the  Instruction, 
surveillance  was  to  bo  a  continuing  effort  of  auditing, 
monitoring,  and  reconciling  the  contractor’s  system  and 
records.  APR  375**7,  he  said,  had  further  defined  the  sur¬ 
veillance  eriort  as  being  similar  to  tbe  demon strati on 
reviews  for  validation,  but  more  limited  in  scope.  The 
same  regulation  assigned  primary  responsibility  for  sur¬ 
veillance  to  Air  Force  Systems  Command  (7.FSC)  .  AFSC,  in 
turn,  assigned  this  function  to  the  Air  Force  Contract 
Management  Division,  to  accomplish  through  the  AFPKO.  The 
result  is  a  surveillrr.ee  effort  accom)  ‘listed  primarily  by 
the  APPRO  or,  if  no  APPRO  is  available,  by  the  coonixant 
Defense  Contract  Administration  Services  Office.  In  cither 
case,  the  Defense  Contract  Audit  Agency  assists  the  sur- 
veilling  activity  whenever  an  aud.i t  capability  is  required. 

Mr.  McDonald  went  on  to  explain  Air  Force  Contract 
Management:  Division  (AFCMD)  Regulation  170—;  v.hich  specified 
ArPRO  res  pons  ibi  1  i  t  ics  and  outlined  tests,  chec'Ks,  and 
reconciliations  to  be  accomplished.  1  h o  pros ram  actual  Iv 
began,  he  said,  before  the  contract  v:a s  even  awarded.  Vhcn 
the  APPRO  received  word  \  be  contractor  was,  designing  n  sys- 
tem  to  be  validated  status.  monitoring  was  begun.  Then, aft  or 
,  l he  APPRO  had  personne 1  on  llv  validation 
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tecnn  and  participated  actively  in  the  validation  review. 

Alter  validation  is  cicco.pl  inhod  *  the  continuing 
surveillance  effort  begins.  Tasks  in  this  area,  in  addi¬ 
tion  to  general  assistance  to  the  contractor,  fall  into 
six  categories.  I  ho  Ai  Cl  ID  regulation  contains  chapters 
prescribing  detailed  tests  and  checks  for  each, 

Reconc ilia t ions .  This  section  requires  implementa¬ 
tion  of  procedures  to  assure  reports  from  the  contioctoi 
reconcile  to  one  another  er.d  to  his  internal  reports-  Also 
specified  arc  checks  to  verify  that  reports  emanate  from  the 
validated  system.  This  insures  the  contractor  does  not 
operate  one  system  for  Government  reports  and  o  di-ferent 
one  for  his  internal  management.  Reports  reconciled  include 
the  Contract  Funds  Status  Report,  Cost  Information  Reports, 
and  Economic  In  forma  L  i  on  System,  reports,  all  of  which  <mc 
generated  from  SAP'S  requirements,  but  arc  not  actually 
a  port  of  C/5JC3C.  ’Ihcy  do,  however,  use  information  from 
the  C/SCSC  system  and  therefore  need  to  be  reconciled 
a  ga i ns t  sy s tem  d  a  ta . 

Organize t i on .  This  chapter  of  the  rcoulo t i on 
requires  the  7.1TR0  to  ensure  the  contractor's  organization 
structure  continues  to  comply  with  the  criteria.  Continuing 
analysis  also  assures  the  bo:  k  breakdown  Structure  is  kept 
current  as  changes  to  work  specifications  occur.  Finally, 
checks  are  proscribed  to  ensure  continued  compliance 


of  contractors*  subsystems,  such  os  hie  scheduling  Qnd  work 
authorization  systems.  If  the  contractor  implements  pro¬ 
cedural  changes  the  Al; PRO  must  reviev/  them  for  compliance. 

Plarining  and  Budgeting.  Hi  is  chapter  provides  guide¬ 
lines  for  ensuring  continued  compliance  with  the  Planning 
and  Budgeting  criteria.  APPRO  personnel  review  the  con¬ 
tractor's  scheduling  and  budgeting  documents  and,  when 
possible,  attend  schedule  and  budget  meetings  held  by  the 
contractor  for  operating  managers.  Observations  of  actual 
beginning  a*'d  ending  dates  are  made  and  later  compared  to 
contractor  records  to  assure  accurate  recording.  The  con¬ 
tractor's  response  to  ahead  or  behind  schedule  conditions 
is  observed  as  is  his  capability  to  identify  and  resolve 
scheduling  problems.  Finally,  spot-checks  are  made  to 
assure  use  of  variance  analysis  to  identify  problems  or 
potential  problems. 

Account! ng.  Tests  prescribed  in  this  chapter  deter¬ 
mine  whether  or  not  the  contractor's  system  continues  to 
comply  with  the  accounting  criteria.  Entries  to  contract 
cost  accounts  arc  verified  to  be  certain  costs  were  actually 
incurred  and  are  chargeable  to  the  contract.  Other  chucks 
ascertain  whether  or*  not  the  costs  are  identified  to  the 
correct  hBS  elements.  Indirect  costs,  such  as  executive 
salaries  and  other  similar  entries  to  overload  accounts, 
arc  checked  to  be  certain  of  proper  allocation  to  contract 
cost  accounts.  This  insures  the  Air  Force  i.  charged  for 
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its  fair  share  of  those  costs  but  not  for  portions  v/hich 
should  be  charged  to  commercial  work • 

Anal ys in.  Provisions  of  this  chapter  require  the 

APPRO  to  assure  continued  performance  analysis  capability 
is  maintained  by  the  contractor  and  to  verify  the  correct¬ 
ness  of  his  analysis.  Various  checks  of  the  system  assure 
correct  identification  of  the  Budgeted  Cost  of  Vtork  Scheduled, 
Budgeted  Cost  of  Work  Performed,  and  Actual  Cost  of  Work 
Performed  for  each  v:ork  package  of  the  17 BS .  Further  checks 
verify  calculated  variances  and  determine  whether  or  not 
explanations  and  follow-up  action  were  adequate. 

Revisions.  The  final  chapter  of  the  regulation 
provides  procedures  the  APPRO  must  use  to  assure  contractor 
compliance  with  the  revisions  criteria.  Evaluations  to  be 
performed  include  checks  of  contractor  handling  of  contract 
changes  to  ensure  timely  and  proper  incorporation  into 
budgets  and  schedules.  Also  included  are  audits  to  detect 
retroactive  changes  to  completed  work  packages,  which  are 
prohibited.  Finally,  procedures  arc  established  to  recon¬ 
cile  new  budgets  to  those  which  existed  prior  to  the  change. 

The  Work  Breakdown  Structure 

At  the  completion  of  the  review  of  the  surveillance 
regulation  Captain  Blake  asked  about  the  relationship 
between  the  UBS  and  the  contractor's  organisation  structure, 
br.  ho  Dona  Id  explained  bow  jx>1  h  the  Wits  end  the  organisation 
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could  bo  represented  by  the  c3n*-.'ir-,i  ~ 

*  ne  cio^‘>icul  organization  charts 

Used  by  many  firms.  ,„„„.  ,„.  soia,  ir  M.;  tuj.nca  olae_ 

vay,  and  overlay*,  o„  the  other.  an  interlocking  matrix  „ould 
be  formed  at  the  lower  level.  *,  illustrate  be  sketched  a 
diagram  (see  Figure  3  on  page  62). 

Tbo  diagram  showed  each  „ork  package  at  tbo  lower 
level  of  the  KBS  within  one  of  the  lower  level  organisational 
elements.  A  given  section  usually  has  more  than  one  work 
package  assigned,  often  from  different  branches  of  the  WBS, 
bnt  each  work  package  is  assigned  to  only  one  section.  ' 

The  work  packages,  Mr,  McDonald  said,  each  represent 
a  ...easureable  unit  of  work  assigned  et  the  level  where  the 
»ork  will  be  performed.  The  package,  are  clearly  delineated 
and  separately  budgeted  and  scheduled.  Then,  when  work 
begins,  actual  costs  are  collected  and  identified  to  the 
proper  work  package.  Tna  contractor  can  then  sum  the  bud¬ 
gets  and  actual  costs  for  work  packages  1.  2.  3.  and  4  in 
the  diagram,  for  instance,  to  provide  totals  for  this  Ele¬ 
ment  A.  These  could  be  added  to  Element  B  totals  to  provide 
figures  for  Element  c  and  so  on.  „p  to  tho  total  Aircraft 

System.  This  suction  is  used  for  reports  to  tho  Cavern- 

ment,  with  figures  rrisortc-  for  n,m  , 

1  LC-  J-ot  tho  top  four  levels  of  the 

>®5.  As  reports  are  forwarded  up  the  government  chain  of 

comm. "ml,  they  arc  further  sum, arise,  until  non  officials, 

as  an  example,  receive  only  o.o  level  one  susmarirntlon. ' 

■ihe  total  Budgeted  Cost  or  a„  work  packages  „„ 

'  be  ooiit  reel  plus  management  and 
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Figure  3 

Example  I'otrix  Formed  by  Interlocking  the 
Contractor 's  Organixat  ion  Structure 
and  the  Work  Breakdown  Structure 
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equal  the  total  contract  price  at  any  given  time.  The  man¬ 
agement  reserves  arc  si. plv  contingency  funds  the  contractor 
is  allowed  to  assign  to  some  management  levels  without 
identifying  thorn  to  a  particular  work  package.  within  the 
limit  of  these  reserves  he  can  cover  small  overruns  on  work 
packages  or  unexpected  increases  in  the  amount  of  work  neces¬ 
sary  to  complete  a  work  package. 

To  go  the  other  direction,  the  manager  of  Branch  1 
can  obtain  reports  on  Section  1  by  totaling  v'.ork  Pac.^cCjes  1 
and  5.  The  manager ' above  the  Branch  might  receive  a  single 
summary  of  all  six  Work  Packages.  Any  manager  may,  of  course, 
ask  for  more  detailed  reports  if  his  summaries  indj-Coce 
problems  are  occurring.  By  analyzing  variances  o  probl en 
can  be  traced  to  the  work  package  creating  it  <-nd  conectite 
action  may  be  initiated. 

Perrorr.ic  nee  Measures 

Since  the  subject  of  variance  analysis  had  orison. 

Hr.  McDonald  began  to  discuss  performance  measures.  Captain 
Blake  was  already  familiar  with  the  Dunce ted  Costs  for  Wcik 
Scheduled  (I3CWS)  ,  Budgeted  Costs  for  V:ork  Performed  (Bcv:p)  , 
and  Actual  Costs  for  Work  Performed  (ACWP)  but  had  not  yet 
seen  how  i  c porting  occurred.  He  was  fhurcioic  shown  P'-tt 
of  the  latest  Cost  Performance  Report  (see  All ochment  4). 

'Jhc  report  displayed  the  BCd.B,  PC’tfP,  and  /.*.WP  both 
for  the  l*i test  period  and  cumulative  to  dal  e-- for  a  third 
level  vm:;  element.  Fourt  h  level  supper,  ling  dal  a  lor  the 


HSS2S 


> 


ki 


64 

entry  also  appeared  on  the  report.  Current  period  and 
cumulative  variances  had  been  calculated  and  explained. 

Ihe  lust  items  on  the  report ,  shown  only  for  the  fourth 
level  item, were  the  approved  budget  and  revised  estimate 
foi  total  cost  of  the  item  at  completion,  as  v/ell  as  the 
variance  between  the  two. 

After  the  briefing  ended  Captain  Blahe  spent  a  few 
dayji,  becoming  better  acquainted  with  operations  at  the 
APPRO  and  the  contractor's. facility.  Then,  having  col¬ 
lected  the  latest  CFSR  and  CPR  from  the  APPRO,  he  returned 
home.  Upon  arrival  he  tool:  .the  reports  to ‘Captain  Bradbury, 
who  told  him  information  would  be  extracted  from  the  reports 
at  the  second  WB3  level  and  reported  to  the  SPO  Project 
Manager.  Significant  variances  would  be  explained  and 
recommendations  made  for  any  fictions  deemed  appropriate, 
inese  actions,  he  said,  could  range  from  conference's 
between  SPO  and  contractor  management  to  initiation  of 
contract-  changes. 

As  the  discussion  ended  Captain  Bleke  returned  to 
his  own  office  feeling  that  he  was  finally  beginning  to 
understand  SAIMK  and  C/SCSC. 

B  i.  s c ur. s  .t on  Q u or  t ion s 


Vi n 1 cy 


c/ncsc, 


niafce. 


When  Captain  Blnhe  returned  to  his  office  Major 
ashed  him  the:  fol.lov.ri  ng  questions  about  SAJMf*, 
and  pcrfoni:  nco  r.ksi:mrcwpn'-..  if  you  were  Captain 
how  would  you  have  answ.  red  them? 
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1.  What  is  1.1  ic  relationship  between  Pits  and  SAIMS? 

2.  VJliat  arc  the  objectives  of  TAXI'S? 

3.  What  arc  the  major  rub- rye  toms  of  SAIMS?  What 
purposes  arc  served  by  each? 

4.  What  arc  the  five  major  areas  of  criteria  in 
C/SCSC?  What  arc  the  primary  requirements  of  each? 

5.  What  arc  the  Validation  and  Surveillance  func¬ 
tions?  Why  are  they  necessary? 

6.  VJliat  is  the  primary  concept  underlying  the  use 
of  the  Work  Breakdown  Structure?  What  benefits  does  the 
VJBS  provide  for  the  Government?  for  the  contractor? 

7*  What  are  the  major  DoD/Aj  r  Force  policies 
governing  administration  cf  S/^IHS  and  C/SCSC?  What  minimum; 
criterion  must  a  program  meet  to  be  subject  to  these  sys¬ 
tems? 

8.  What  conditions  evicted  after  World  War  II  and 
up  to  the  early  1960s  that  led  to  the  development  of  SAIKS 
and  C/SCSC? 

9.  Suppose  that  you  were  given  the  information 
shown  below  for  an  clement  of  the  WBS.  What  variances 
would  you  calculate  from  the  informat  ion?  What  is  the 
arnemnt  of  the  variances?  What  do  they  indicate  about  the 
WI3S  element? 


Hem 


/♦.mount 


Thidgoted  Cost  of  tori:  Tithed id  ed 
Budgeted  Cos.  I  of  VJoi  >:  Per  f  orm*  d 
Actual  Cost  of  Woi. k  l’('i:ion.K\l 


$  25,000 
23,000 
24,500 


G6 


Assume  that  the  figures  are  all  cu  ;ula live  through  the 
latest  reporting  period. 
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Suyges t ed  Answers 

1.  Resource*  Management  Systems  (RMS)  is  the  name 

given  to  the  program  begun  in  1906  when  Secretary  of 
Defense  McNamara  issued  DoD  Directive  7000.1.  rihis  program, 
designed  to  create  integrated  financial  management  in  the 
DoD,  is  composed  of  four  major  systems:  (1)  the  Programming 
and  Budgeting  System  (PBS) ,  (2)  the  Operations  Management 

System  (OPS),  (3)  the  Inventory  Management  System  (IMS), 

and  (4)  the  Acquisition  Information  and  Management  System 
(AIMS) .  The  latter  two  are  assets  management  systems,  with' 
IMS  serving  for  internal  asset  management  information  and 
AIMS  for  external.  AIMS,  in  turn,  consists  of  the  Selected 
Acquisition  Management  and  Information  System  (SAT MS)  which 
covers  certain  selected  major  acquisitions,  and  Other 
Capital  Acquisitions,  for  the  remainder.  SAIMS  is  there¬ 
fore  a  sub-system  of  one  of  the  four  major  RMS  systems. 

2.  SAIIJ3  has  several  major  objectives  which,  if 
met,  make  it  advantageous  to  both  the  Government  and  to  the 
contractor.  First,  SAIMS  is  designed  to  provide- an  ade¬ 
quate  basis  for  responsible  decision  making  by  oath  con¬ 
tractor  management  and  the  responsible  DoD  component. 

Second,  SAJMS  is  designed  to  minimise,  to  the  extent 
practicable,  the  data  gathering  and  reporting  workload 
imposed  on  contractors  and  in-houso  Don  activities.  Third, 
SA.TMS  has  an  objective  of  identifying  management  control 
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systems  and  procedures  which  ere  most  effective  in  meeting 
contractor's  requirements  and  encouraging  acceptance  and 
installation  of  those  systems  and  procedures.  Finally, 

SAIMS  has  an  objective  of  providing  integrated  financial 
management  reporting  of  usable  summary  data  for  all  eche¬ 
lons  of  internal  management.  Meeting  the  latter  two  objec¬ 
tives  allows  SAIMS  to  meet  the  first  two  by  providing  the 
vehicle  for  their  accomplishment. 

3.  The  four  major  sub-systems  of  SAIMS  are  the 

i 

Economic  Information  System  *(ni3)  ,  Cost  Information  Reports 
(CIR) ,  Contract  Fund  Status  Reports  (Cl SR) ,  and  the  Perform¬ 
ance  Measurement  System  (PMS) .  The  primary  purpose  served 
by  the  EIS  is  to  provide  data  for  analyzing  the  impact  of 
defense  spending  by  industry?  and  by  geographic  area-  The 
EIS  repor is  arc  furnished  to  Congress,  Dob,  and  other 
interested  Federal  agencies.  Cost  Information  Reports  are 
provided  to  the  various  Dob  c<  ponents  to  furnish  necessary/ 
data  on  estimated  and  actual  costs  of  the  acquisition  cycle 
of  completed  programs.  Dutc  from  the  CIR  arc  used  to  build 
a  consistent  data  base  for  cost-os timat  ‘ing,  programming, 
budgeting,  and  procurement  of  future  systems.  Contract 
Funds  Status  Reports  are  furnished  to  the  contracting  agency 
through  the  AFPRO  and  SFO  to  provide  the  necc.ss  ry  data  for 
determining  funding  requirements  by  fiscal  year  for  the 
remainder  of  the  existent  progm’11-  The  Perform.- no  Me.  n:ro- 
inont  System,  encompass  j  ng  cost,  schedule*,  and  per¬ 

formance  measurement  areas,  also  furnishes'  c  ;tn  for  the 
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SPO.  It  is  controlled  by  the  Cost/Schedule  Control  System 
Criteria  (C/SCSC)  and  Systems  Engineering  r.anaqemc-nt  (SKM) . 
Ihe  former  provides  data  which  enables  the  Government  ^nd 
contractor  to  maintain  visibility  into  the  contractor's 
progress  and  position  with  respect  to  cost  ana  schedule 
requirements  of  the  contract.  5FJ-1  provides  visibility  into 
the  technical  aspects  of  the  contractor’s  performance. 
Together  the  four  sub-systems  provide  a  complete,  integrated 
system  for  control  of  major  acquisitions. 

4.  Hie  major  sections  of  the  Cost/Schedule  Control 
System  Criteria  and  their  requirements  are: 

a.  The  organization  criteria,  which  require  ^hc 
contractor  to  define  the  required  word.,  sog— 
rnent  it  into  v.'oitjC  prsclcugefs,  and  assign  c«ch 
v;or*k  package  to  a  single  organizational  ele¬ 
ment  within  his  facility  for  accomplishment. 

b.  ihe  planning  er.d  budgeting  criteria,  v.hich 
require  the  contractor  to  schedule  and  bud¬ 
get  for  each  v:or3;  package  so  actual  perform¬ 
ance  can  later  be  comp;  roo  to  the  pi o n  ior 
each  individual  veil:  pad; age . 

c .  The  n  ccount i n g  c r  i  t e r .1  a ,  vh .i  ch  r c  qu i  r c:  th c 
contractor’s  accounting  system  to  be  ade¬ 
quate  to  record  both  direct  and  indirect 
costs  applicable  to  the  contract  and  identify 
costs  to  the  appropriate  work  p ichages . 
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d.  The  analysis  criteria,  which  require  the  con¬ 
tractor's  system  to  provide  monthly  figures 
on  applied  costs,  budgeted  costs  for  v.'orh 
scheduled,  budgeted  costs  for  work  performed, 
actual  costs  of  word:  performed,  and  current 
estimates  of  the  total  cost  at  contract 
completion,  these  criteria  also  require 
the  contractor  to  calculate  and  anolyze 
variances  and  use  the* a  as  the  basis  to  ini¬ 
tiate  actions  to  identify  and  correct  prob¬ 
lems  which  created  the  variances, 
e.  The  revisions  criteria,  which  require  the 
contractor  to  establish  adequate  procedures 
to  assure  continuity  of  budgets  end  plans. 
Ihose  procedures  must  result  in  contract 
changer  being  budgeted,  scheduled,  and  inte¬ 
grated  into  the  existing  plan  so  that*  the 
original  plan,  plus  or  minus  changes,  can 
be  reconciled  to  the  ncv.T  plan-  Inherent  in 
this  is  the  necessity  to  assure  avoidance  of 
retroactive  changes  to  budgets  and  costs  for 


c o: apl c ted  v/orl; . 

5.  The  Validation  fund. ion  is 
team  of  r, overman!.  personnel  conduce  a 
review  of  the  structure  and  operation 
management  information  system  *»nd  nuuv» 


a  piece-:.4’  vlurcby  a 
do  t  a  i  J.  cd  )  >or  f  -aval  ru 
of  the  contractor's 
gemont  <>}  :rnt  ions • 


This  on-si tc  demonstration  review  of  actual  operations 
enables  team  members  to  insure  operation  of  an  effective 
cost/schedule  planning  and  control  .system,  determine  its 
compliance  with  the  C/SCSC,  and  develop  an  understanding 
of  the  way  the  system  is  designed  and  how  it  worts.  It 
also  provides  a  vehicle  for  training  Government  personnel 
in  the  uses  of  performance  measurement  data  by  giving  them 
the  opportunity  to  observe  the  collection  and  analysis 
methods  used  by  the  major  government  contractors. 

The  Surveillance  function,  carried  out  primarily 
by  the  Defense  Contract  Administration  Service  Office  or 
Air  Force  Plant  Representative  Office  serving  the  contract 
consists  of  reconciliation  of  data  on  contractor  reports 
and  checks  to  assure  continued  contractor  compliance  with 
the  various  criteria  of  C/SCSC.  This  function  is  carried 
out  primarily  after  completion  of  Validation  and,  unlike 
Validation,  is  a  continuing  effort  to  assist  and  monitor 
contractor  operation  of  the  system. 

6.  The  underlying  concept  of  the  V'ork  Breakdown 
Structure  (V7BS)  is  Performance  Measurement.  A  common  base 
is  needed  to  allow  integration  of  the  work  required  by  the 
contract  with  the  contractor's  organizational  structure, 
'ibis  is  realized  by  performing  success;* vo  sub-divisions  of 
the  required  work  until,  at  the  lowest  level,  the  division 
represent  separate  units  of  work.  Much  of  there  is  clo< rl 
identifiable  and  may  be  separately  budgeted  and  scheduled, 
by  assigning  each  work  package  to  a  single  organizational 


clement  for  accompli shment ,  the  contractor  creates  an  inter¬ 
locking  matrix  with  the  '..TiS  on  one  axis  and  his  own  organi¬ 
zational  chart  on  the  other,  Uy  controlling  the  v:orX  at 


this  level,  it  is  a  simple  matter  to  summarize  data  by 

r 

combining  V7H3  elements  at  successive  work  levels  and  provide 

0 

data  compatible  with  the  contract  work  specifications  for 
Government  use.  Conversely,  it  is  easy  to  combine  those 
VJDS  elements  v/hich  are  assigned  within  given  organizational 
elements  to  provide  data  compatible  with  the  needs  of  man¬ 
agers  at  any  given  level  of  the  contractor's  organization. 

Thus ,  the  system  is  able  to  provide  data  to  both  parties  in 
the  form,  and  at  the  level  of  summarization,  most  helpful 
to  them.  .As  an  additional  benefit  for  both  parties,  the 
level  of  control  established  allows  timely  discovery  of 
variances  from  plan  and  facilitates  identification  of  prob¬ 
lems  which  led  to  those  variances.  Unfortunately,  this 
system-- as  any  other- — focuses  attention  on  the  problems  but 
does  not  solve  them.  That  is  still  up  to  the  managers  in¬ 
volved  and,  as  always ,  they  may  or  may  not  follov;  through 
on  the  problem  solving. 

7.  n AIJ’S  and  its  component  sub-systems,  including 
C/SCSC ,  tare  applied  to  acquisitions  estimated  in  the  Five 
Year  Defense  Plain  to  require  total  cumulative  Research, 

Development,  Test,  and  Evaluation  financing  in  excess  of 
$25  million  or  cumulative  production  investment  in  excess 
of  $100  million  (except  Finn  Fixed  Price  contracts).  In 


WS&SUXks'. 


73 

applying  the  systems  to  acquisitions  meeting  these  criteria, 
contractors  are  not  required  to  use  any  specified  manage¬ 
ment  control. system  or  organize  in  any  specific  way.  Rather, 
the  contractor's  system  is  to  be  of  his  own  design,  subject 
to  the  minimum  acceptability  requirements  of  the  criteria. 
Capability  of  the  contractor's  system  to  satisfy  these 
criteria  may  be  considered  as  a  factor  in  evaluating  his 
response  to  a  Request  for  Proposals  but  not  to  the  exclusion 
of  weapon  system  design  considerations,  proposed  price,  and 
other  evaluation  factors.  Finally,  during  performance,  of 
a  contract  requiring  C/SCSC,  all  performance  data  furnished 
by  the  contractor  must  be  taken  from  the  internal  manage¬ 
ment  control  system  validated  by  the  Government.  Each  of 
these  policies  is  intended  to  leave  the  contractor  free  to 
design  a  management  information  system  capable  of  satisfying 
Government  requirements  for  information  and  internal  require- 

merits «, 

8.  The  basic  conditions  which  led  to  the  develop¬ 
ment  and  implementation  of  SAIkS  and  C/SCSC  began  during 
and  after  World  War  II.  During  the  period  up  to  the  time 
of  their  development  the  cold  war  created  a  situation  in 
which  a  standing  military  force  was  needed.  Concurrent 
technological  advances  entailed  arming  this  force  with 
successively  more  complex  and  more  expensive. weapons  sys¬ 
tems.  The  ensuing  production  of  large-  numbers  ci  complex 
systems  unique  to  the  militajy  resulted  in  the  establish¬ 
ment  and  growth  of  companies  oriented  to  defense  production. 
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As  the  dollar  value  of  defence  contracts  grow  the 
DoD  competed  for  increasing  shares  of  the  Federal  tax  dol¬ 
lar.  Predictably,  pressure  began  to  mount  for  better  and 
better  management  of  the  defense  share  of  the  tax  dollar. 
This  pressure  increased  as  the  complexity  of  v;eapons  sys¬ 
tems  made  it  more  difficult  to  estimate  accurately  in 
advance  what  the  cost  of  a  new  system  v/ould  be. 

In  response  to  pressures  from  Congress  and  the 
public,  DoD  established  and  tried  numerous  systems  or  con¬ 
trol  to  improve  management  in  thus  area.  Some  met  with 
initial  success  but  v;ere  la^er  misapplied  to  non-similar 
programs  with  disastrous  results.  In  addition,  defense 
contractors  began  to  complain  cf  being  forced  to  implement 
redundant  systems,  many  of  which  were  not  appropriate  to 
their  management  and  or ga nr x a t ror.a  1  structure. 

In  response  to  this  situation  AFSC  oesigned  che 
basic  system  adopted  by  DoD  as  C/SCbC,  the  desired  result 
of  which  is  to  provide  both  DoD  end  contractors  with  the 
information  needed  to  manage,  but  without  preempting  the 
contractors*  right  to  design  a  management  information  and 
control  system  tailored  to  organisational  requirements. 

9.  The  two  variances  to  bo  calculated  from  the 
given  information  are  the  schedule  variance  and  the  cost 
variance.  Tho  schedule  varinn  e  is  calculated  by  subtract¬ 
ing  the  budgeted  Cost  of  toil  idchectulcd  from  the  Budgeted 
Co:> L  of  Work  Performed: 

(UCWP  -  D'MS)  -  $?d,ooo  -  v:*s , ooo  -  C\^ono 
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This  negative  variance  is  unfavorable  nnd  indicates  that 
work  is  behind  schedule  on  the  WBS  clement.  The  cost 
variance  is  calculated  by  subtracting  the  Actual  Cost  of 
Work  Performed  from  the  Budgeted  Cost  of  Work  Performed: 

(BCWP  -  ACWP)  =  $23,000  -  $24,500  -$1^500 


This  variance,  also  unfavorable,  indicates  actual  cost 
incurred  in  performing  the  vxork  was  greater  than  the  bud- 

gated  amount  for* the  .same  vjov~k. 

In  both  cases,  the  circumstances  causing  the  vu r i - 
ahee  should- bo  identified  and  appropriate  action  taken  to 
explain  the  variance.  If  the  problem  is  of  a  continuing 
nature,  action  should  also  bo  taken  to  resolve  it  before 


further  schedule  slippages  and  cost  overruns  occur. 
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COST/ EC  t I EDULE  CONTROL  system  criteria 

The  following  criteria  ore  taken  from  Enclosure  1  to 
Del)  Instruction  7000.2  dated  December  22,  X9C7.  The  cri¬ 
teria  c^re  listed  an  Paragraph  3  of  that  enclosure  and 
constitute  pages  3-6  of  the  original  cocur.ent . 

Criteria 

The  contractor's  system  v;ili  include  policies,  pro¬ 
cedures,  and  methods  which  are  designed  to  -insure  *-nat  it 
will  accomplish  the  following: 

a  *  Organ  j  zati  on 

(1)  Define  all  the  authorised  work  and  related 
resources  to  meet  the  requirements  of  the  ccr.uract,  using 
the  framework  of  the  contractor's  extension  of  an  appro¬ 
priate  work  breakdown  structure# 

(2)  Identify  the  authorized  work  within  the 
f ollowing  categories : 

(a)  Discrete  work  packages  with  a  defined 

end  result,  or 

(b)  Level  of  effort  or  opr  nr  1 1  cr.ed -off  or  t 
work  packages  whose  completion  c«oes  not  prot.rcc  a  definable 

end  result. 

(3)  Identify  the  internal  or  ~  ele¬ 

ments  and  the  major  subcontractors  respm  -lc  i  o'  accom¬ 
plishing  the  authorized  work# 

(4)  identify  the  managerial  '  "tie?.:*  lespon— 

siblc  for  controlling,  overhead  (indirect  costs). 

(9)  Identify  overhead  (indirect  costs)  and 
the  methods  used  for  its  allocation. 

b .  Planning  _  and  Puck  fed  i  jxj 

(])  Describe,  plan  and  rduahle  l ho  work. 

Alt  .eh:.  :;t  1  (1  of  4) 
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(2)  Identify  physical  products,  nj 1 os tones, 
technical  performance  goals,  or  otner  indicators  thtit  will 
he  used  to  measure  output. 

(3)  Establish  budgets  for  all  authorized  v:orh . 

(4)  To  the  extent  the  authorized  work  has  been 
identified  in  the  categories  described  in  3. a. (2)  above, 
establish  budgets  for  those  categories  in  terms  of  dollars, 
hours  or  other  acceptable  units. 

(5)  Establish  overhead  budgets  for  the  total 
costs  of  each  significant  organizational  component  whose 
expenses  will  become  indirect  costs.  Reflect  in  the  con¬ 
tract  budgets  at  the  appropriate  level,  the  amounts  accumu¬ 
lated  in  overhead  pools  that  will  be  allocated  to  the 
contract  as  indirect  costs. 

(6)  Identify  * management  reserves,  if.  used. 

(7)  Provide  that  the  contract  prices  plus  the 
estimated  urtdef initized  price*  of  authorized  but  unpriced 
changes  and  unpriced  work  is  reconciled  with  the  sum  of  ail 
internal  contract  budgets  and  management  reserves. 


(3)  Retain  the  original  budgets  for  those  ele¬ 
ments  of  the  work  breakdown  st r  vice  arc  iccnti —  2  ea  t:s  p^  ieea 
lino  items  in  the  contract  and  for  -chose  at  the 

lowest  level  of  the  Dot)  '’reject  Purr  ary  ••orb  i’roakdovm 
Structure  as  a  traceable  basis  against  which  contract 
performance  can  be  compared. 


c .  Accounting 

(1)  Record  applied  direct  costs  on  a  basis 
consistent  with  the  budgets '.in  a  formal  system  thuu  is 
controlled  by  the  general  boohs  of  account. 

(2)  Record  indirect  costs  nil  or  part  of  which 
vrill  be  allocated  to  the  contract. 

(3)  These  formal  records  in  (1)  and  (2)  above 
should  make  it  possible  to  determine  unit  or  ioi*  costs  tor 
pr i c ed  1 ine  i t ems . 

( 4 )  H un r. ;,i r iso  a ] ■»)  0  i  ed  d  i re r. t  r a s t s  n nd  ov <.  r - 
head  allocations  in  the  ;  : counting  records  fo>-  (a)  those 
elements  of  the  work  breakdown  si  lhrdurc  ic3c  rMficd  as- 
priced  line  items  in  the  contract,  ;  r-(!  (b)  those  elemrnis 
at  Mr  1o\.v:;I  lovol  on  the  1X>!»  Mrojccl:  riwvij-  v;oi:k  kvccV- 
down  Structure. 


ii:  ,  !  I  (2  of  -5) 
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(5)  Identify  the  bases  for  a 11 oca 
of  level  of  effort  or  apportioned -of fort  v:or); 
appropriate  cost  accounts* 


ting  the  cost 
packages  to 


(b) 

i ncu  r red  cos  t  s , 
costs) - 


Provide  a  ban 
applied  direct 


is  for  auditing  records  of 
costs;  and  overhead  (indirect 


d. 


Reporting 


(1)  Identify  on  a  monthly  basis  or  jriore  often 
at  the  discretion  of  the' coni rac. or  in  the  ^tail  neeued 
by  raanacj enient  for  effective  control,  u.n,,  e^ta  fio.r, 
reconcilable*  with,  the  accounting  .,y».»uC'.iu 

(a)  Applied  direct  costs  for  worl.  performed 
and  the  budgeted  costs  for  the  same  v/ork. 


indirect  costs* 


(b)  Actual  indirect  costs  and  budgeted 


for  work  performed  and 


(c)  Budgeted  cos t.r 

budgeted  costs  for  vork  scheduled.  . 

(d)  Sianificr-nt  variances  resulting  fro:?, 

the  above  comparisons  classified  in  t orris  of  ' 

overhead,  and  any  other  appi  opt*r«- «.e*  el <•... -i.-~ >  -  --- 

the  reasons  therefor . 

(2)  Identify,  or.  i  monthly  basis  or  rore  often 

r  .  •  _  r  onir  t  dii-eicnv.cs 

at  the  discretion  o*.  u.j  co...2.<  :^  ~  -r^rr-l  -■*-  rO.-v,™ 

between  actual  and  p3.cnr.ca  sc.ucjSi^  ^ 
technical  performance. 


(3)  identify 
neccssa ry  by  the  above. 

c .  Rev 5  : \ ions 


•  rd  planned 

.eg ether  with  the  reasons  therefor, 
naoorial  actions  that  are  nedo 


(1)  noth  :atc.  the  of  Co 


,/ji.i i  author i r.c d 


changes  and  int.crn.nl  rop^a,.r  ^  0,-‘  ,-nc  rCrfraci, 

formonce,  schedule,  :n:e  co.'a  p.  . ' .  f-'  r.ng 

and  record  the  effects',  or  i-u.horrSJea 
replanning  actions-  in  s'.chcuvut-ss  ana  nu,;a  ... 

(2)  Reconcile  orJcincl  bu  ivCd  s  fo>-  fl-O-  e  oU. 
went  s  of  the  v.-osh  breakdown  .  'r^rc^  ;  -  "  the 

line  iter.ss  in  l he  con.  rvc <  ,  -..u  -0.  •;  ,.nkdov:n 

lowest  level  Of  the  I'oD  iiojm  •  V  O' ,."t  (  )  >v 
JtruoU.ro.  with  current  budge  « r-  sn  of  (•  )  on 
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to  the  authorized  v/orh  and  (b)  internal  replanning  in  the 
detail  needed  by  management  lor  effective  control. 

(3)  Prohibit  retroactive  changes  to  records 
pertaining  to  v:orh  performed  that  v/ill  cuan^c  previously 
reported  amounts  for  applied  ciiioct  costs,  inaircct  costs, 
and  budgets,  cr.ce  ot  for  normal  accounting  adjustments  or  for 
reasons  agreed  to  by  due  contracting  parties. 

(4)  based  on  performance  to  date  and  on  esti- 
mates  of  future  conditions,  develop  latest-  revised  esti¬ 
mates  of  cost  at  completion  and  reconcile  tnese  with: 

(a)  Original  budgets  for  those  elements 
of  the  tor);  Breakdown  Structure  identified  as  priced  line 
items  in  the  contract, 

(b)  Original  budgets  for  those  elements 
at  the  lowest  level  of  the  DoD  Project  Summary  ”ork  Break- 
d o v/n  Str uc  t ur e 

(c)  Current  budgets, 

(d)  Contract  price. 


(e)  The  cent 


fund  requirements  reported  io  the  Government 


ictor’s  latest  statement  oi 
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Figure  4 

h  Typical  Work  Breakdown  Structure  (Partial)** 


*Breahdovn  not  .shown 

**Adapio:1  from  partial  v:orl:  breakcocr:  rvru'iure 
shown  in  “Point  i;  of  CO.'PA SS,  "  undated  brio!  ry  ::oith  Ameri¬ 
can  Bochv/ell,  fnc.  ,  p*  5. 
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The  following  exerpt  from  Section  II  of  the  R-l 
Report  of  Cost/Schedule  Control  System  Damons trot ion  in 
the  verbal  description  supporting  the  affirmative  determin¬ 
ation  for  Demonstration  Review  Checklist  Item  l.a. 

Organisation 


Da,  in  response  to  Contract  Data  Item  K-126  (Con¬ 
tract  Work  breakdown  Structure)  and  Statement  of  t'ork 
P  ragraph  2.1.B.1,  'lorth  ;.:.or i  can  has  developed  end  is 
maintaining  a  Contract  Work  Breakdown  Structure  (  bnS; 
end  Dictionary  that  complies  with  provisions  set  forth 
in  Mtl-Std-fiOl.  The  final  negotiated  CUDS  CS.-70-79) 
v;as  submitted  to  the  SPO  on  1  duly  1970.  The:  summary 
levels  (levels  one  through  three)  of  the  CV;BS  a  no  the 
Dor tli  American  extension  to  the  cost  account  level  of 
detail  have  been  -reviewed  and  approved  by  !.q  AH>C. 
Exhibit  1-1  is  the  first  of  twenty-four  paces  that 
comprise  the  contractual  C..B3  Inuex. 
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Cost  Performance  Report  En>: 
(Dollar:;  in  Thbusar.de) 
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Appendix  C 

t 


V/^L ID  AT  10! :  IV/J I! .’AT  IO! Z 
RELATED  INSTRUCTION 


FACULTY  GROUP  INSTRUCTIONS 


The  attached  case  study  is  the  prime ry  output  of  my 
research  paper.  It  is  intended  to  present  the  Cost/Schedule 
Control  System  Criteria  (C/CCSC)  to  students  in  the"  Finan¬ 
cial  Menu cement  course  of  the  Graduate  Logistics  program. 
However,  it  is  n oc  the  intent  of  either  the  case  or  the 
course  to  present  a  detailed  review  of  the  individual 
criteria.  Rather,  the  case  is  designed  to  shov:  hov;  the 
C/CCSC  operate  as  a  concept,  hov:  they  form  a  basis  for  per¬ 
formance  near urn. vent,  and  how  the  “cLlcncar-of -events"  might 
look  for  an  application  of  the  criteria. 

One  of  the  objectives  of  my  research  poper  is  to 
test  the  validity  of  the  resultant  case  as  a  teaching  aid. 
The  method  I  have  chosen  to  do  this  entails  admir.i stration 
of  an  examination  to  two  croups  o‘f  officers.  One  croup, 
composed  of  students,  has*  already  tab  on  the  exam.i  rati  on 
based  on  knowledge  of  the  subject  matter  gained  in  the 
Financial  ! :anac;o.:.ent  course.  I  an  new  asking  an  equal 
group  of  faculty  officers  to  cover  the  case  study,  and  take 
the  same  examination.  In  addition  to  the  examination 
results  themselves,  I  am.  also  inheres  Led  .in  any  comments 
— pro  or  con — ->u  i:  y  have  aooui:  the  case.  A  four  question 
evaluation  sheet  is  therefore  included  for  your  use. 

Accordingly,  I  rccr  st  that  you  read  the  attached 
case,  go  throngs,  the  discussion  questions  and  s user:  ted 
answers  to  them,  and  give  seme  thought  to  the  material 
presented  in  then.  Having  b one  o  please  take  the  twenty— 
five  question  multiple  choic a  examination.  Please  take 
this  cis  a  " c los od -nook M  exam.  Then,  please  answer  the 
evaluation  questions,  adding;  any  comments  you  deem:  appro¬ 
priate,  and  return  this  package  to  ;  c.  You  are  welcome  to 
place  the  case  copy  and  examination  in  my  mailbox  "anony¬ 
mously”  if  you  so  choose.  As  you  go  through  l he  case, 
please  feel  free  to  annotate  correction:*  to  grammar,  punc¬ 
tual  t  j on ,  spo  1 1  i ng ,  etc.,  if  you  c : re  so  j  r. c .1  iiV . j . 

Thank  you  for  your  time  and  effort.  1  recognise 
ho,*T  busy  you  arc  at  this  Lima  and  appreciate  your  ooop:  ra¬ 
tion. 

D^jj}  JO 

DONALD  1).  \\R: \C:VV,  Captain,  UfiAF 
Gin  i  Log  Clo.ss  71*1 
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_P louse  advise  me 
of  my  score. 


NAME_ 

RANK 


student  niKnr.R 


* 


The  attached  examination  con.; 5  sis  of  twenty-five 
multiple  choice  questions  concerning  the  Cost/Schedule 
Control  System  Criteria  and  ret  ted  ropier..  Each  question 
has  only  one  correct  answer.  Indicate  your  choice  of  the 
answers  by  a  ci rc.lo  or  other  mar};  arcur.c;  the  letter  in 
front  of  the  answer.  Please  answer  each  question  to  the*  host 
of  your  Knowledge.  if  you  do  not  Know  the  answer  to  a 
question  DO  POT  guess.  Your  score  will  r.ot  be  Known  to  any¬ 
one  other  then  myself  ar.d  I  am  interested  in  your  knowledge 
in  this  area,  not  in  your  ability  t  o  guess. 

Please  indicate  your  .name,  rat’d:,  and  stude  nt  num¬ 
ber  in  the  sprees  at  the  top  of  this  pace.  Tf  you  would 
like  to  know  how  you  score  on  the  exam, :  rk  the  statement 
at  the  above  left.  ‘You  ::  v  ) jc g i n  wherever  you  aie  ready. 
There  is  no  time  limit  to  complete  the  exam  so  t«..he  your 
time. 


rJhe  following  abort  vii  Lions  are  used  throughout  the 
examination; 
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1.  The  Delected  Acquit  it;  ion  Information  and  Management 
System  (SA.'MS)  is  boat  described  as: 

a.  A  sub-system  of  the  “ Programming  and  Budgeting 

System11  portion  of  the  Resource  Management  Systems 

(RMS). 

b  A  sub— system  of  the  "Operations  Management  System 
portion  of  RMS. 

c.  A  sub- system  of  the  "Inventory  Management  System" 
portion  of  RMS. 

d.  A  sub-system  of  the  "Acquisition  Information  and 
Management  System"  portion  of  RMS. 

e  Not  a  part  of  RMS.  SAIMS  is  a  separate  but  comple¬ 
mentary^  system  intended  to  perform  the  same  function, 
but  for  external  rather  than  internal  information. 


* 


2.  SAIMS  is  intended  for  application  to: 

a.  All  acquisitions  by  the  Dob  from  external  sources. 

b.  Those  acquisitions  specifically  designated,  on 
individual  basis,  by  the  Secretary/  cl  Defense, 
Secretary  of  the  Air  Force  (or  clhor  service),  or 
the  Chief  of  Staff  of  the  procuring  service. 

c.  Those  acquisitions  estimated  in  the  Five  Year 
Defense  Plan  to  require  total  c nmole  civ o  Research, 
Development,  Test  and  evaluation  rinancinc  in 
excess  of  $25  million  or  cumulative  production 
investment  in  excess  of  $300  million  (except  Firm 
Fixed  Price  contracts) . 

d.  Certain  critical  acquisitions  from,  contractors 
whoso  past  performance  has  be- on  below  <  oecg  e 
levels  but  who  occupy  a  sole-source  position  on  a 
mu  j  or  wcu pon  sy r;  1 «.  ra . 

e.  Acquisitions '.of  i  jer  v.v;  pon  syrUms  which  •  ic 
estimated  to  exceed  $10  mil  ion  total  cost, 
including  b^th  Ros.o*  nroh ,  bevel oj <" nl  ,  Icr't  *ad 
F.vol  ua  Lion  and  prodncl  ion  cords. 
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3  Uhich  of  the  following  is  not  an  objective  of  the 

Selected  Acquisition  Information  ana  Management  ^s- 
ter.i  (SA IMS)  ? 

a  To  bring  to  the  attention  of  and  encourage  DoDcon- 
tractors  to  accept  and  install  management  control 
systems  and  procedures  which  are  most  effective 
in  inoetincj  tlioir  rotiuircinontss. 

To  provide  an  adequate  l»sis  for  responsible 
decision  malting  by  both  contractor  :r.,:nage...e..t  on! 
DoD  component's. 

To  provide  a  standard  internal  planning  and  control 
system  with  a  uniform  chart  of  accounts  for  use 
by  DoD  contractors  in  order  to  csov.ro  co,..pati.jilx  1 
and  accuracy  of  information  from  then. . 

To  minimize,  to  the  extent  practicable,  the  data 
gathering  and  reporting  v.orhlotu  imposed  on  con- 
tractors  and  in-houso  activities. 

To  provide  an  integrated  financial  mantgemen*- 
reporting  oyster,  which  will  provide  -ufc-7 

data  for  all  echelon:  of  internal  sauna gemenu. 


b. 


c. 


d. 


e. 


4. 


The  four  irsior  sub-nyt  Vo.t.:;  of 
Incarnation  and  linnaeciaont  r  y, 


th c-  G c  1  e c t cd  7. cq uisir.5  or. 
ton  (FAIFS)  arc: 


a  • 


Economic  In  tor:  action  Fysl’  c  . 
Gy  13 ton.  Contract  Funds  Star 
a  nee  I-'cecurencn  t  (in  c:  ludinn 
technical)  . 


,  Opcr-  tlons  Knraccrxmt 
us  Uc 'port/  and  Poriorr..- 
cos t ,  schea ul a  and 


b. 


c. 


a. 


O. 


Contruct  Fund  Status  noport 
Sy f ;  to: r. ,  Port orvnr-  nc  c  I  * ca  s  u  r c 
r,  ch  c  d  u  le,  a  n  d 
Reports • 


pc  o'/  :oni  c  In  f  oxt.V2  t  x  on 
nt  ( i  nc  1  ud  :i. r.o  cos  t , 
technical)  ,  and  Cost  Jniorr.,. uion 


rei  fom ncc  Idoatunrcv.c  nt  (xncluairc;  c 
and  technical)  ,  Asset:'  Eanacjc: ’.cnt  Fa 
for.ua t.i on  Reports,  and  Prograr.  .me  - 
Systeiu. 


i  ,  .schedule, 
r  >,  Post  An- 
i'uacati  no 


Prorira:*:  Opera t  i  one.  Fys.t«;i,  PO  i  oi 
(including  cost  /  s.cbc ciulo,  **r  1  \* 

JnLorr.vrti.on  Reports,  and  Pont  met  * }  vsu 
Report. 


1) 


aim 

,  Cost 
l  stu 


t 


CijyA.  in  ora  it  ion  F  per’,  s,,  Fonlrmt 

Report,  inventory  «;<  n  .c'  out  Fysi* 
unco  !->*a:;\jrc;  onl  (including  co  , 
t  Cfin*  i  ca  1 )  . 


Funds  Flatus 
'  ;  ivrfonn- 
hfuulc,  and 


ife,i 


wpi  iiiwjjfatwfr  ifrafe  ai:..* _  „  .  n£mt* 
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5.  The  suh-system  of  SAIKS  v;hich  provides  data  for  deter¬ 
mining  funding  requirements  by  fiscal  year  is: 

a.  Contract  Funds  Status  Report* 

b.  Economic  Information  System* 

c.  P r  og r  aim;  i.i no  and  Budgeting  System* 

d.  Operations  .Management  System* 

e.  Cost  Information  Reports. 


6.  The  sub -system  of  SAIKS  which  provides  data  for  analysis 
of  the  impact  of  defense  spending  by  industry  and  by 
geographical  area  is: 

a.  Cost  Information  Reports. 

b.  Contract  Funds  Status  Reports. 

c.  Cost  and  Economic  Status  Reporting  System. 

d.  Economic  Information  System. 


o.  Programming  and  budgeting  System. 


f 

M 


7.  The  sub -system  of  S7.j”S  which  provides  data  on  actual 
and  estimated  costs  of  the  acquisition  cycle  of  com¬ 
pleted  programs  to  built,  a  consistent  date  base  for 
cost-estimating,  programming,  budgeting,  and  procure¬ 
ment  of  future  systems  is: 

a.  Performance  Measurement  (including  cost,  schedule, 
and  technical) . 

b.  Programming  and  Budgeting  System. 

c.  Contract  Funds  Statu:;  Report. 

» 

d.  Operations  Management  System. 

e.  Cost  Information  Reports. 
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8.  Under  3AJK3  the  Pcifomnnce  Measurement  sub -system 
includes  the  CoiJi /Schedule  Control  Bystein  Criteria 
(C/3C.3C) ,  consisting  of  five  major  areas  of  criteria. 
Those  live  are: 

q m  Organization,  Planning  and  Budgeting/  /-.ccounting. 
Analysis,  and  Revisions. 

b.  fork  Breakdown  Structn  ,  Organization,  Analysis, 

Rev  is  ions,  a  nd  Ac  c  o  un  t  mg. 

c.  Planning  and  Budgeting,  Accounting,  Auditing, 
Analysis,  and  Revisions. 

d.  Planning,  Budgeting,  Accounting,  Analysis,  and 
/aid  i  tiny. 

e.  Organization,  Planning  and  Budgeting,  Accounting, 
Analysis,  and  Validation. 

9.  The .  portion  of  the  C/SC30  Which  is  concerned  v.Tith 
defining  the  required  v;orh,  breaking  .it  into  verb 
packages,  and  assigning  each  work  package  to  a 
particular  section  within  the  contractor’s  organi¬ 
zation  is* 

a.  Reporting  Criteria. 

b.  Work  Breakdown  structure  Criteria. 

c .  Orga  ni sat i oi  Cri ter ia . 

d.  Accounting  Criteria. 

e.  Work  Assignment  Criteria, 

10.  'the  portion  of  the  C/8C3C  which  is  concerned  with  assur¬ 
ing  that  identifiable  work  is  scheduled  and  rue  •  tea  for 
:.ur:h  -.It; it  actual  performance  can.  b-ter  be  coapaicu  to 

the  phin  is: 

a .  Orynni  zat.i  on  Cri t  cria  r 

1).  Planning  and  lUidyek.iny  Cri  tori 

c.  VJork  Breakdown  3 huriurt  Cri  \  o  h'. 

Audi  ting  Cr Ll ^r ia. 


d. 
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i7  ..... * 
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’  V. 

i 


r 


11  The  purposes  of  C/SCSC  validation  of  a  contractor's 
'  internal  planning  and  control  system  include: 

To  insure  an  effective  operating  cost/ schedule 
planning  and  control  system. 


a. 

b. 

c. 

d. 

e. 


To  develop  understanding  of  the  planning  ana  con¬ 
trol  systems  used  by  major  contractors  and  to 
determine  their  complin nee  with  the  C/^CoC. 

To  develop  a  .trained  staff  of  personnel  vho  under¬ 
stand  contractors'  management  operations  ana  the 
use  of  performance  near u-re.men t  date. 

'i 

All  of  the  above. 

'‘a."  and  nb.  “  above,  but  not  "c." 


12. 


Final  C/SC3C  Validation  action  (approval  or  disap¬ 
proval)  is  based  primarily  on: 


a . 


Review  of  the  system  deser 
contractor  as  part  of  his 
for  Proposals,  along  vztn 
Defense  Contract  Audit  Age 


iption  submitted  by  the 
response  to  the  Request 
reviev  of  the  attached 
r cy  c o: aments . 


b. 


c. 


An  in-depth,  data  !e 
in  which  a  val  Ida  tic 
facility  and  ohsoimm 
a  period  after  «’-vr<- 

Results  of  the  bei^r 
of  the  operating  :  y: 


c*-C...on;:t  ration  of  the  system 
tea:,  visits  the  contractor’s 
acral  system  operation  for 
of  the  contract. 

:e  Contract  Audit  A.gency  review 
-Cr.  * » 


d. 


e. 


Tnc  cent rector’s  c s t a i 1 cd 
ten  to  sr*0  and  pr<  an 
place  before  contract  av: 


presentation  of  the  sys- 
t  personnel,  v.*l i i f 'h  t a no s 
rd. 


Recommend  a  f  ioi ' : 
Re  pres  on  fa  t  ivo 
Service  Office 
system  Cor  the* 
av.a  rd . 


o'-  r-'  rc-ridcs.l-  Air  Force’ Hr-nt 
m  !)-,lor.r<  "or.-;  r.-.-L  ;  m  i;.  i :  '  r.V,ior. 
t. hr. t  re  v i c v:c o  t. tic  ' r u 1 n; * ct < n 
rov(:rs  ent  sul  eerne  ut  to  contract 
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15.  The  Validation  process  for  u  contractor* a  system 

n  o  rmo  1 3  y  oc  o  u rs r 

a.  Before  lie  is  allowed  to  submit  proposals  on  pro- 
grams  requiring  the  use  of  C/SCSC. 

b.  After  l he  contractor  submits  a  proposal  on  a  pro¬ 
gram  which  would  require  the  use  of  C/S  CSC,  but 
before  lie  is  awarded  a  contract. 

c.  When  he  is  identified  as  having  submitted  the  lev; 
proposal  in  response  to  a  Request  for  Proposals 
involving  C/SCSC.  Tf  he  is  otherwise  eligible, 
the  contract  award  is  temporarily  withhold  until 
his  system  is  validated.  Pa i lure  to  become 
validated  within  ICO  days  will  result  in  award 
being  m  de  to  the  next  lowest  contractor  with  a 
validated  system. 

d.  After  receipt  by  the  contractor  of  a  contract 
award  requiring  the  use  of  C/SCSC.  . 

e.  Kach  time  he  receives  a  new  contract  requiring  the 
use  of  C/SCSC. 


#  16.  The  SA..T.MS  SurveiJ  K  nee  Program  is  c  rried  out  primarily 

by : 

a.  The  System  Program  Of  !  ire*  (SPO)  . 

b.  The  Defense  Con’ m  rt  /\udit  7.grnry,  with  assistance 
from  the  ArLC  S awe .  i  1  once  Office. 

^  c.  The  cognizant  Defense  Contract  Adnirlst ration 

-f  .Service  Office  o*~  Air  Poire  Plant  Representative 

Office,  valth  assistance  from  the  Defense  Contract 
.  Audit  /tgoncy. 

d.  Hi0  nearest  7.ix~  vorce  Systems  Co;  a  r  f  Contr*  ''tor 

*  Survoi  11  r  nee  Office,  vaith  s;a.sb  nc<  f  >  cm  the 

Defense  Contract  /.edit  Arm  ncv. 


c . 


4 


* 


Rone  of  the  ot/>vo. 


( 
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17.  The  Cost /Schedule  Control  System  Criteria  is  extended 

to  cover  sub-contracts  of  a  qualified  programs 

a.  Only  if  the  sub- con tractor  has  a  previously 
validated  system. 

b.  At  the  election  of  the  prime  contractor,  who  is 
then  responsible  for  validating  the  sub-contractor. 

c.  Cover.  C/CCSC  applies  to  Government  contracts 
only,  not  to  contracts  between,  the  contractor 
and  his  :rub- con  tractors. 

d.  V.Tic-n  selected  by  the  System  Pro  era::.  Office  ana 
prime  contractor  according  to  i*he  criticality 
of  the  sub-contract  to  the  prog  rein. 

e.  Only  v;hcn  the  sub-contract  meets  the  same  criteria 
as  the  basic  contract. 

18.  Which  of  the  following  is  not  characteristic  of  the 


v;o 

ik  pa  eh no os  developed 

under  the  V' 

oik  hr 

cakdov/n 

Structure? 

a . 

They  rc 

present  un 
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;  of  work  at 

level 

s  where 

work 

is  peri 

ermed. 

b. 

"heir  v: 

ovk  oontor 

t  :i 

s  clearly  0 

c 3 i net 

tc  i  f rc 

m  all 

oilier  v; 

or);  pack  .<« 

c. 

The  sum 

of  the  b- 

i ce  costs  o 

i  all 

work  pm 

Ck-  CO: 

is  cqua 

1  to  the  t 

of  r. 

j  cor.tr o”l 

price. 

d. 

Tnny  are  arsicn*;}: 

5.1c 

to  a  si nolo 

opc  Tv 

time  or 

car:!- 

nation 

for  coco: re 

d  i; 

e. 

They  }  in 

vo  budgets 

cr¬ 

;  pro area  in 

t  errs 

c  coll 

;  r  ^  ^ 

nr.n-hou 

ra,  oi  oth 

ier 

measurable 

uni t a. 
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* 


J 


103 


4 


19.  The  Surveillance  Program  lor  C/SCSC  includes  all  of 

the  following  except: 

a.  A  program  of  reconciliation  testing  on  the  data 
on  contractor  reports  to  the  Government. 

b.  Chech s  to  assure  that  all  retroactive  chances  to 
completed  work  packages  are  fully  documented  and 

valid. 

c.  Checks  to  assure  that  the  cord: rector 1  s  organi¬ 
zation  structure  continues  to  comply  with  the 
C/SCSC  Organizational  Criteria. 

d.  Checks  on  the  contractor's  capability  to  calculate 
and  use  variance  .information  to  locate  and  resolve 
problems. 

*  e.  "Checks  to’ assure  'the  propriety  of  changes  which 
affect  tlie  contract  budget  baseline. 


i 


% 


t 


20.  then  the  Validation  demonstration  review  results  in 

finding  discrepancies  in  the  contractor  itanagoinent 

s  y s  t cm  the  va  3.  i  d  a  t  i  on  l: earn : 

a.  Advises  the  Pyste:.  Program  Office  {.'IPO)  to  with¬ 
hold  award  of  l hr  contract. 

b .  Advi  s es  tl  ie  Gy.  t  c:: :  •  e;  : g  r  am  Office  to  be g  i  r.  a  ct i  on 
to  terminate  J  lie  contract  for  cause. 

c.  Is  authorized,  according  to  the  C/SC  PC  provisions 
of  the  contract  ,  to  i  dec  such  changes  to  the 
contractor ' s  system  as  are  required  to  meet  minimum 
roqu  i  remnant  s  of  tl  ie  cr  1  tens . 

d.  Advises  the  contractor  that  failure  to  make  neces¬ 
sary  changes  to  assure  co.  pi:-  n*c  v:ithln  a  period 
of  90  days  will,  result  in  less  of  the  contract. 

e.  Submits  a  final,  re  port  of  their  f i:  \  ys  which, 
after  approval  r  i a  l.n-nmi  tr  «  to  the.  co nt  rector 
so  tli  a  L  he  m  *  y  ben  nv  r  •  •  VJ  •*  y  -he  n<  or-;  ss  ry  cl  ia  net  r 
to  as  sine  comp l i  nnce. 
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21.  The  C/SCSC  Surveillance  Program  cons in tn  of: 

a.  A  semi-annuol  revi c*v;  of  the  contractor's  system, 
records,  and  documents,  performed  by  the  cognizant 
survci  11  ing  activity,  along  w ith  assistance  to  the 
contractor  as  requested. 

b.  A  Scheduled  annual  review  of  the  contractor's 
system,  records,  and  documents  plus  one  unan¬ 
nounced  audit  of  flic  system  each  year,  both 
perf onst/d  by  the  cognizant  survci  11  ing  activity 
and  accompanied  by  assistance  to  the  contractor 
as  requested. 

c.  Direct,  and  continuing  participation  in  the  con¬ 
tractor's  management  prcoe*5;s  by  a  selected  team 
from  the  surveilli  rej  office,  tho  members  of  v;hich 
are  assigned  to. work  in  th.e  contractor's  facility. 

d.  A  continuing  effort  of  audit,  monitoring,  and 
reconciliation  of  the  contractor’s  system,  records, 
and  documents  by  trie  cognizant  curve .i  1 1  ing 

.  .  activity,  along  wiih  assistance  to  the  contractor 
as  requested. 

o.  None  of  the  above. 

22.  Which  of  the  fo.llovai nq  is  r/r Lai.se  of  the  hark  Break- 

dowi i  otru a* Lure? 

a.  To  enable  Lho  contra  * ter  to  integrate  the  resultant 
work  packages  .with  his  organizational  structure  a ud 
thereby  i  den  t  i  i  y  ( •  cl  i  erg  a  n  :i  :a  i  5  one  1  c]o:  r.c nt  v; ■»  th 
the  v;ork  it  must  support. 

b.  To  facilitate  budgeting  and  srhcclulirc  of  the 
indivldu.  1  elements  arraigned  to  o.ceh  o:  rn-.ni- 
z a  t  i  ona  3  demon  t . 

c.  To  enable  tho  contractor  and  Cover nr.anc  personnel 
to  .separately  ident  ify  ar.J  control  X  h  sc  v;cr);  park- 
aqcs  which  wore  a v;  rck  d  to  the  conti a  ;or. 

d.  To  enable  the  coni  met  or  to  identify  .*  nd  analyze 
s.i  gni  f.i  cant  variants  m  jrol  rac*i  •■•rs  causing 
cost  oyer  runs  one  under  runs  or  cord  r.'m  >ni  i  ng  to 

s  chc  d  u  1  c  s )  \  ppr .  qos . 

d.  "b."  and  "d."  above. 
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23,  Which  of  t ho  following  is  a  valid  statement  of  Air 

Force  policy  concerning  C/SCSC? 

a.  Application  of  the  criteria  v:ill  r.ot  be  interpreted 
an  requiring  the  use  of  specified  management  con¬ 
trol  systems. 

b.  Performance  da ta  furnished  by  the  contractor  will 
be  taker  from  the  contractor's  internal  manage¬ 
ment  control  system. 

c.  Application  of  the  criteria  v.nll  not  be  interpreted 
as  requiring  contractors  to  organize  or  reorganize 
in  any  specific  way. 

d.  The  capability  of  the  ccrh ractor *  s  internal  manage¬ 
ment  control  system  to  satisfy  the  C/.q^SC  may  be 
considered  as  an  clement  in  evaluating  his  response 
to  an  individual  Request  for  Fro}x>sal. 

e.  All  of  the  above. 


24.  Which  of  the  following  is  true  of  the  c/fhf 
Criteria? 


7-ccoun  t  i  r.c 


a.  They  require  the  contractor's  system  to  bo  adequate 
to  record  both  efi  j  :t  and  :  r.d  irc-ct  cost:-:  appli'  hole 
to  the  contract. 

b.  They  specify  the-'  lh'  stars .  id  ]  oD  chart  of  accounts 
is  to  be  utilize  j  .1  r.  trie  contractor's  rysfem. 

c.  They  require  the  cor.4  rector 1  s  system  to  be  able 
to  identify  all  crpl liable  contract  costs  to  the 
appropriate  elements  of  the  vrork  breakdown  Structure. 

d.  They  require  the  contractor 1  s  system  to  ix*  suf¬ 
ficient.  ly  accurate  to  incut e  id. at  actual  cc:  tr: 
do  not  vary  appreciably  fro;:  budget  *  a  re  ts. 

e.  "a."  and  ,!c.  ,r‘ above. 


i 

* 


25. 
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Si  Svc^^'nt  =Uin,  «.*  collet™  by  MO 
mcint  system? 

a.  Organization  Criteria. 
t>.  7*.c,coun  ting. Criteria. 

c.  Analysis  Criteria. 

d.  Revisions  Criteria. 

e.  Hone  Of  the  above,  C/SCSC  does  not  require  reports. 


* 


t 


r  c 


J 


_ 1 

*m-.  ~  " ;  ’  r  ’  ^  '  ~- 


Cose  Evaluation 


After  you  have  coinplotad  the  attached  examination/ 
please  provide  answers  to  the  rollowing  questions*  it  is 
not  necessary  for  you  to  relent  try  yourself  on  this-  sheet. 
However,  if. you  would  cure  to  do  so  it  would  enable  me  to 
consult  you  for  clarification  of  any  cor.enonts  you  make  on 
this  sheet.  Thank  you. 


1. 


Do  you  feel  that  this  case  accomplished  the  desired 
learning  objectives  listed  at  the  beginning  of  the  ease? 
If  your  answer  .is  “no,  “  please  list  specific  areas 
where  you  felt  the  case  was  weak. 


2.  VJhot  specific  suggestions  would  you  make  for  improving 
the  case  study? 
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T  3.  How  long  d id  it  take  you  (approximately)  to  cover  the 

care,  including  reading  time  and  time  spent  answering 
#  tlio  question:;  posed  at  the  end  of  the  cuseV  Exclude 

time  spent  on  the  examination. 

i 

t 


<  ^ 

J 


4.  Do  you  feel  that  case  coverage  of  the  topic*  of  the 
questions  at  the  end  of  the  can o  was  .sufficient  t> 
enable  the  student  to  answer  the  question:  without 
undue  difficulty?  If  not,  please*  c:i  to  specific 
instances  of  weakness. 
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frnwi.  ra  to  Exn:ni nation  Ouer 


The  following  Indicated  respi.jes  are  the  correct 
ancv'err  to  the  examination  quer  ti  crx;. 


Oaor tion 

1 

2 

3 

4 

5 

6 

7 

'  8 
9 

10 


Correct 
Rcnpor:  c 

d 

c 

c 

b 

a 

d 

c 

a 

c 

b 


11 

12 

13 

.14 

15 


d* 

b 

a 

e 

d 


16 

17 

18 

19 

20 


c 

d 

c 

b 

e 


21  d 

22  *  c 

23  e 

24  c 


25 


2  . 
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Appendix  I ) 


METdOD  FOR  CAI/*’JI.ATXOR  Or  CU'-l’iA' -  '>  J- UU  FIG* 
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METHOD.  FOR  CALC U1AT JOS  OF  CUMULATIVE  STEP  FUNCTION 


The  purpose  of  this  Appendix  as  to  describe  the 
technique  used  to  calculate  the  cumulative  step  function 
values  displayed  in  Table  2,  pace  23. 

Symbols  used  in  the  calculation  ^re  defined  to  be: 


tjiiiupli.? 

croup  one. 

the  cioup  which  f 

;tudi cd 

c/scsc 

1  \*  »  » * 4“,  nf 

- - 

the  care  study. 

S  ia.pl  e 

group  fv;o/ 

the  group  which  . 

;t\.Oied 

C/SCSC 

in  the  Fir. 

nncial  M.  nc.cjea.out 

course 

X  “  A  random  variable,  the  scoro  of  n  ?/  her  of 
c  i  ther  g r oi; n  on  the  .  e:*:a:  i  r.a t  ion . 


K  = 


S„. 00 


n2 


Tr .  o  r.  u:  a  her  o  ~ 

scores  c.f  oi  * 

:  he  r  . 

arc*  lea:;  than 

a  civoc:  valu< 

a  of 

Vno  observed  c 

v  .  a  m.  .  :  va  :>  l  *, 

jO  fu 

group  ore.  [. 

v  fX;  :  K/r.i 

] 

The  cbr’/TV'.  '-  y 

1  -  ;  ;  V  ;  t  • 

r)  fv# 

group  t  .  :o .  [ 

,  (a)  •  h/u.j 

] 

of 


D  =•  The  maxi.:  .u: .  ob.*crv"d  cit>  rt  rf,c  b'twcou 


S„  (/)  ar/.i  Cm  (>•)  in  the  p»-<  d  i  clcu  oi  notion. 
nl  ,-2 


Si  nco  the  j  >:<•’  of  f  .ch  r;  ale  v.v*:  {  »  r,  11.*  »;eno:  an¬ 
atom  for  Sn  (;*:)  am  Sn^  (X)  v.v  rr  both  t  en.  Fox  puip'*'e:  0l- 


1  2 

the  test,  the  range  of  poard  *:0  **  more  van  divide  i  iHo 
ten  citnul  dive  in  otgr.c’r.tr;  of  l<*n  point:  c «  :h,  ;vnh  that 
increment  ntr  consisted  of  .'msmrer.  less  1  an  tea.,  i  nor*  a<r-r*t 
tv;o  of.  sco res  less  thus:  tv:onty,  c >  r' .  'il'S  .  i  ./rater  of  *  I *<- 


U  1 
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two  fractions  was  then  determined  by  ecu: 
of  a  cjroup  which  fell  in  to  each  cunm?*t. 

The  difference,  in  the  predicted 
between  the  tv/o  Craceicns  for  each  i  ncrc; 
by  subtraction. 


r>i  f  f  ei  r-ncc  ~  Sn  (X)  -  Sr 

i  ^ 

Ihe  in- ix i lutcu  dif fere-nee  VeS  found  by  on..*.' 
to  !'1)#  the  critical  D  value  for  sampler 
discussed  in  Clidpter  3. 


-ting  the  scores 
i ye  i  nereuent  • 

6  direction, 

:e n t  v;a  s  caleulat od 

(X) 

i vet ion  and  compared 
of  sice  ton,  as 


